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PREFACE 

This  report  was  conducted  as  part  of  the  Evaluation  of  Environmental  Investments  Research  Program 
(EEIRP).  The  EEIRP  is  sponsored  by  Headquarters,  U.S.  Army  Corps  of  Engineers  (HQUSACE).  It  is 
jointly  assigned  to  the  U.S.  Army  Engineer  Water  Resources  Support  Center  (WRSC),  Institute  for  Water 
Resources  (IWR),  and  the  U.S.  Army  Engineer  Waterways  Experiment  Station  (WES),  Environmental  lab 
(EL).  Mr.  William  J.  Hansen  of  IWR  is  the  Program  Manager,  and  Mr.  H.  Roger  Hamilton  is  the  WES 
Manager.  Program  Monitors  during  this  study  were  Mr.  John  W.  Bellinger  and  Mr.  K.  Brad  Fowler, 
HQUSACE.  The  field  review  group  members  that  provide  complete  program  direction  and  their  District 
or  Division  affiliations  are  Mr.  David  Carney,  New  Orleans  District;  Mr.  Larry  Kilgo,  Lower  Mississippi 
Valley  Division;  Mr.  Richard  Gorton,  Omaha  District;  Mr.  Bruce  D.  Carlson,  St.  Paul  District;  Mr.  Glendon 
L.  Coffee,  Mobile  District;  Ms.  Susan  E.  Durden,  Savannah  District;  Mr.  Scott  Miner,  San  Francisco  District; 
Mr.  Robert  F.  Scott,  Fort  Worth  District;  Mr.  Clifford  J.  Kidd,  Baltimore  District;  Mr.  Edwin  J.  Woodruff, 
North  Pacific  Division;  and  Dr.  Michael  Passmore,  Walla  Walla  District.  The  work  was  conducted  under 
the  Incorporating  Risk  and  Uncertainty  Into  Environmental  Evaluation  Work  Unit  of  the  EEIRP.  Mr.  L. 
Leigh  Skaggs  of  the  Technical  Analysis  and  Research  Division  (TARD),  IWR  and  Mr.  Richard  Kasul  of  the 
Natural  Resources  Division  (NRD),  WES  are  the  Principal  Investigators. 

The  work  was  performed  by  The  Greeley-Polhemus  Group,  Inc.  (GPG)  under  Task  Order  No.  1, 
Contract  No.  DACW-72-95-D-0002,  managed  by  Mr.  Leigh  Skaggs.  Dr.  Charles  Yoe,  a  principal  of  GPG, 
was  the  principal  author. 

Valuable  review  comments  and  suggestions  were  received  from  Mr.  Bob  Bass,  Galveston  District; 
Mr.  Bob  Blama,  Baltimore  District;  Mr.  Bruce  Carlson,  St.  Paul  District;  Ms.  Pat  Cory,  Baltimore  District; 
and  Mr.  Steve  Garbarino,  Baltimore  District. 

The  report  was  prepared  under  the  general  supervision  at  IWR  of  Mr.  Michael  Krouse,  Chief,  TARD; 
and  Mr.  Kyle  E.  Schilling,  Director,  IWR;  and  at  EL  of  Dr.  Robert  M.  Engler,  Chief,  NRD  and  Dr.  John 
W.  Keeley,  Director,  EL. 
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EXECUTIVE  SUMMARY 

Incorporating  risk  and  uncertainty  analysis  into  environmental  restoration  planning  studies  is  seen 
as  a  means  of  improving  the  quality  of  the  decision-making  process.  This  report  introduces  Corps  personnel 
involved  in  the  planning  of  environmental  restoration  projects  to  the  basics  of  risk  and  uncertainty  analysis. 
In  addition,  it  presents  a  sample  incorporation  of  risk  analysis  in  the  application  of  a  habitat  evaluation 
procedure,  habitat  suitability  index  model. 

Environmental  planning  has  been  done  by  the  Corps  for  a  long  time.  The  growing  national  interest 
in  environmental  issues  has  created  a  new  emphasis  on  environmental  values  within  the  Corps’  program. 
This  new  emphasis  brings  with  it  new  initiatives.  Environmental  planning,  at  one  level,  is  nothing  more  than 
the  application  of  good  planning  techniques  to  environmental  problems  and  opportunities.  Although  this 
report  tends  to  focus  on  environmental  restoration  projects,  the  principles,  tools  and  methods  discussed 
throughout  are  equally  applicable  to  any  environmental  planning  activity. 

Environmental  planning  is  future-oriented  problem  solving  and  decision-making  intended  to  restore, 
preserve,  protect  and  enhance  environmental  resources  or  to  mitigate  unavoidable  damages  to  them.  As 
such,  it  is  fundamentally  decision-making  under  uncertainty.  Incorporating  risk  and  uncertainty  analysis  into 
environmental  planning,  wherever  it  is  done  in  the  organization,  can  improve  both  the  planning  process  and 
the  quality  of  decisions  made  to  manage  the  nation’s  environmental  resources. 

Risk  analysis  is  practiced  by  analysts  from  many  disciplines.  As  a  result,  there  is  not  a  single 
broadly  accepted  taxonomy  of  terms.  The  terminology  used  by  Corps  analysts  in  environmental  restoration 
is  introduced  in  a  simple  manner.  For  example,  risk  is  defined  intuitively  as  the  chance  that  something  bad 
happens. 

Risk  and  uncertainty  analysis  is  easier  to  introduce  to  new  applications  when  there  is  a  structured 
and  logical  approach  to  the  process.  There  are  three  broad  types  of  uncertainty:  knowledge  uncertainty, 
model  uncertainty  and  quantity  uncertainty.  Quantity  uncertainty  is  the  most  commonly  encountered. 
Understanding  the  nature  of  the  uncertain  quantity  can  go  a  long  way  toward  prescribing  a  treatment  for  the 
uncertainty.  Of  all  quantities,  empirical  quantity  uncertainty  can  be  the  most  complex  to  address,  because 
there  are  at  least  seven  different  causes  for  these  types  of  uncertainty.  The  taxonomy  offered  in  this  report 
provides  the  new  risk  analyst  with  a  way  to  think  about  the  uncertainty  that  is  present  in  environmental 
planning  work. 

The  time  may  come  when  risk  and  uncertainty  protocols  for  environmental  restoration  studies  are 
described  in  detail  in  Corps’  guidance  and  the  models  and  tools  needed  to  conduct  these  analyses  are  well 
known  to  all.  That  day  has  not  yet  arrived  and  so  the  report  presents  a  list  of  selected  tools  and  broad 
concepts  used  by  many  risk  analysts  in  Chapter  Four.  Described  in  the  report  are  the  basic  concepts  of  which 
environmental  planning  risk  analysts  should  be  aware,  including:  acknowledging  uncertainty,  education  and 
training,  EPA’s  ecological  risk  assessment  paradigm,  full  documentation,  jargon  and  literature,  peer  review, 
post  facto  evaluation,  publicity,  risk  communication,  and  software.  These  tend  to  address 
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information/communication  issues.  The  more  specific  tools  described  include:  analytical  solutions,  decision 
criteria,  decision  trees  and  influence  diagrams,  expert  opinion,  probability,  probability  distributions,  random 
sampling,  sensitivity  analysis,  simulations,  specialty  models,  and  statistics. 

Just  as  there  are  broad  concepts  that  can  be  used  in  risk  and  uncertainty^^  analysis  there  are  “big 
picture”  issues  of  uncertainty  encountered  in  environmental  planning  studies.  These  issues  are  not  unique 
to  environmental  planning.  In  fact,  they  are  more  indigenous  to  the  planning  process  than  they  are  to  the 
environment.  Uncertainties  are  encountered  in  each  step  of  the  planning  process  and  they  can  become 
cumulative  if  not  addressed,  virtually  assuring  that  decision-makers  labor  in  ignorance  of  potentially 
important  issues.  Nonetheless,  the  uniqueness  and  complexity  of  environmental  plans  suggests  that  the 
uncertainties  inherent  in  the  planning  process  may  become  particularly  acute  in  environmental  planning. 
These  generic  sources  of  uncertainty  are  discussed  on  a  planning  step-by-planning-step  basis  in  Chapter  Five, 
where  knowledge  and  use  of  the  Principles  and  Guidelines  six-step  planning  process  is  suggested  as  a  means 
of  reducing  uncertainty. 

In  addition  to  the  big  picture  uncertainties,  there  are  many  uncertainties  specific  to  environmental 
planning.  Four  major  sources  of  uncertainty  include;  project  performance,  extrapolation,  models  and  the 
habitat  evaluation  procedures.  Common  potential  sources  of  uncertainty  include:  delineation  of  the  study 
area;  identification  of  target  species;  the  construction  of  habitat  suitability  index  models;  habitat  variable 
measurements;  calculation  of  existing  and  future  habitat  units;  and  modeling  project  performance  in  the 
habitat  evaluation  procedure. 

An  example  introducing  risk-based  analysis  to  the  estimation  of  habitat  unit  changes  is  offered  to 
demonstrate  the  feasibility  of  some  of  the  methods  presented  in  the  report.  A  simple  example  is  presented 
in  Chapter  Six  illustrating  how  risk-based  analysis  can  be  used  to  address  some  of  the  uncertainty  inherent 
in  the  habitat  evaluation  procedures  analysis  that  is  so  common  to  environmental  restoration  projects. 
Selected  results  from  this  analysis  are  presented  to  demonstrate  the  types  of  outputs  that  can  be  obtained 
from  a  risk-based  analysis. 

The  example  shows  it  is  easy  to  use  spreadsheet-based  Monte  Carlo/Latin  Hypercube  simulation  to 
introduce  risk  analysis  into  the  estimation  of  habitat  suitability  indices  and  habitat  units  by  simply  varying 
key  data  inputs.  The  results  of  some  habitat  suitability  index  computations  are  found  to  be  extremely 
sensitive  to  the  habitat  suitability  index  model’s  structure.  In  addition,  the  example  suggests  that  risk-based 
analysis  could  be  come  a  valuable  formulation  tool  insofar  as  it  helps  analysts  understand  the  relative 
importance  of  the  uncertain  variables.  The  simple  example  establishes  the  feasibility  of  using  a  risk-based 
approach  to  habitat  unit  estimation  with  no  significant  increase  in  study  costs  or  schedule  and  confirms  its 
potential  as  a  tool  that  can  help  improve  the  quality  of  environmental  restoration  planning  decisions. 
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CHAPTER  ONE;  INTRODUCTION 

INTRODUCTION 

The  U.S.  Army  Corps  of  Engineers’  Civil  Works  program  has  historically  evolved  and  changed  to 
meet  the  changing  needs  and  priorities  of  the  Nation.  Two  relatively  recent  changes  in  the  Corps’  program 
are  of  particular  Interest  in  this  report.  They  are  the  increased  emphasis  on  environmental  outputs  of  existing 
and  new  projects  and  the  increasing  use  of  risk  and  uncertainty  analysis  in  the  Corps’  decision-making 
processes. 

The  trends  toward  greater  emphasis  on  environmental  outputs  and  more  use  of  risk  and  uncertainty 
analysis  began  at  different  times  and  for  different  reasons.  Now,  as  environmental  activities  are  routinely 
undertaken  by  planning,  operations,  engineering  and  construction  divisions  throughout  all  Corps  districts 
and  risk  and  uncertainty  analyses  have  reached  a  level  of  maturity  and  acceptance,  there  is  a  confluence  of 
these  two  trends. 

The  national  interest  in  risk  and  uncertainty  analysis  had  its  genesis  in  the  analysis  of  environmental 
risk  analysis  in  the  late  1960s.  The  National  Environmental  Policy  Act  of  1969  (NEPA)  is  generally  credited 
with  beginning  the  interest  in  risk  and  uncertainty  analysis.  It  stands  to  reason  therefore,  that  now  that  the 
Corps’  involvement  in  environmental  activities  is  reaching  a  critical  mass,  the  Corps  should  begin  to 
introduce  techniques  of  risk  and  uncertainty  analysis  into  its  decision  process  in  order  to  improve  the  quality 
of  decisions. 

Introducing  risk  and  uncertainty  analysis  to  the  Corps’  environmental  activities  is  a  new  initiative. 
It  will  require  personnel  to  learn  a  few  new  tricks.  But  the  introduction  of  risk  and  uncertainty  analysis  to 
this  area  of  endeavor  will  not  impose  new  significant  burdens  on  analysts  or  managers.  It  may  be  as  simple 
a  matter  as  using  all  the  information  you  have,  rather  than  some  of  it.  It  may  involve  playing  a  few  “what 
if’  games  during  the  decision  process.  Over  time,  as  the  value  of  risk  and  uncertainty  analysis  is  proven  in 
the  decision  process,  it  may  prove  helpful  to  develop  specific  tools  or  adopt  more  sophisticated  analytical 
techniques. 

This  report  explores  the  potential  for  introducing  risk  and  uncertainty  analysis  into  environmental 
planning  activities  in  much  the  same  way  the  Guidelines  for  Risk  and  Uncertainty  Analysis  in  Water 
Resources  Planning,  IWR  Report  92-R- 1  (Principles)  and  92-R-2  (Example)  did  for  water  resources  planning 
generally.  It  begins  with  the  simplest,  least  effort  and  most  effective  ways  to  improve  the  quality  of 
decision-making  in  the  Corps’  growing  environmental  planning  activities.  Those  who  learn  to  use  risk  and 
uncertainty  analysis  well,  and  early,  will  reap  the  benefits  of  improved  decision-making  through  the  better 
informed  development  of  more  robust  plans. 
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THE  PURPOSE  OF  THIS  REPORT 

The  purpose  of  this  report  is  to  introduce  Corps  planners  and  other  interested  parties  to  the  concepts 
of  risk  and  uncertainty  that  already  are  or  may  soon  become  useful  in  improving  the  quality  of  decisions 
made  in  the  formulation  and  evaluation  of  environmental  investments.  The  reasons  for  doing  this  now  are 
straightforward:  I)  risk  and  uncertainty  analysis  contributes  to  better  decision-  making;  ii)  the  Corps  is 
already  making  extensive  and  effective  use  of  these  techniques  in  other  programs;  iii)  there  is  growing  public 
demand  for  environmental  risk  analysis;  and,  iv)  there  is  a  reasonable  likelihood  that  risk  analysis  will 
become  a  legislative  requirement  in  the  not-too-distant  future.  The  objectives  of  this  report  are  simple.  First, 
it  explains  what  is  meant  by  environmental  planning.  Second,  it  provides  a  brief  introduction  to  the 
“concept”  of  risk  and  uncertainty  analysis.  Third,  it  provides  an  introduction  to  some  of  the  basic  tools  and 
methods  of  risk  and  uncertainty  analysis.  Fourth,  it  integrates  the  concepts  of  environmental  planning  and 
risk  and  uncertainty  analysis  through  a  generic  identification  of  potential  sources  of  risk  and  uncertainty  in 
the  Water  Resources  Council’s  six-step  planning  process  used  by  the  Corps.  Finally,  it  introduces  some 
potential  tools  and  methods  that  can  be  used  specifically  to  do  risk  and  uncertainty  analysis  in  environmental 
planning,  with  particular  emphasis  on  the  habitat  evaluation  procedure  (HEP).  It  is  not  a  procedures  manual 
nor  does  it  prescribe  a  specific  approach  for  any  situation.  It  is  hoped  that  the  introduction  to  the  concepts, 
tools,  and  methods  of  risk  and  uncertainty  analysis  will  stimulate  creativity  in  their  application  that  will 
ultimately  lead  to  formal  analytical  approaches  and  tools. 

INTENDED  AUDIENCE 

The  basic  audience  for  this  report  is  “Corps  planners.”  This  includes  personnel  assigned  to  planning 
divisions  and  others  as  well.  Environmental  activities  are  being  conducted  in  various  locations  throughout 
the  Corps’  organization.  Planning  divisions  conduct  Section  1135  and  other  planning  studies.  Operations 
personnel  are  looking  for  beneficial  uses  for  dredged  material  and  conducting  their  regulatory  work. 
Environmentalists  can  be  found  throughout  the  organization  doing  analyses  for  various  elements  of  the 
Corps’  program.  All  of  these  people  are  making  decisions  that  affect  the  environment  and  the  future.  As 
such,  they  are  doing  planning,  though  they  may  not  recognize  or  consider  what  they  do  as  “planning”.  It  is 
to  the  people  who  do  this  kind  of  work,  regardless  of  the  division,  section  or  branch  to  which  they  are 
assigned,  that  this  report  is  targeted. 

ORGANIZATION  OF  THE  REPORT 

The  report  is  organized  to  achieve  the  above  objectives.  In  the  next  two  chapters,  the  basic  concepts 
of  environmental  planning  and  risk  and  uncertainty  analysis  are  presented.  This  is  done  to  develop  the  scope 
of  this  report  and  to  introduce  some  of  the  basic  terminology  it  uses.  The  chapter  on  environmental  planning 
indicates  that  this  is  not  really  so  new,  although  the  increasing  emphasis  on  environmental  restoration  may 
be.  The  risk  and  uncertainty  chapter  defines  basic  terminology,  discusses  the  reasons  for  doing  risk  and 
uncertainty  analysis  in  more  detail,  and  offers  a  taxonomy  of  uncertainty  that  may  help  the  reader,  new  to 
this  area,  to  think  about  uncertainty  in  their  domain  of  expertise. 
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The  fourth  chapter  presents  an  introductory  overview  to  a  number  of  broad  concepts  and  specific 
tools  that  can  be  used  to  address  the  sources  of  risk  and  uncertainty  identified  in  subsequent  chapters.  The 
next  two  chapters  integrate  environmental  planning  and  risk  and  uncertainty  analysis.  Chapter  5  presents 
higher  order  sources  of  uncertainty  in  environmental  planning.  They  are  called  “higher  order”  because  they 
are,  to  a  great  extent,  generic  sources  of  uncertainty  that  might  be  found  in  any  planning  activities.  They  are 
illustrated  and  presented  with  an  environmental  flavor  in  the  context  of  the  Corps’  six-step  planning  process. 
Chapter  6  presents  some  sources  of  risk  and  uncertainty  that  are  specific  to  environmental  planning  activities 
and  a  sample  application  to  the  use  of  a  habitat  suitability  index  (HSI)  model  in  a  HEP  analysis.  The  report 
concludes  with  a  summary  and  some  recommendations  about  which  sources  of  risk  and  uncertainty  can  be 
addressed  first  in  order  to  begin  to  incorporate  risk  and  uncertainty  analysis  into  environmental  planning 
activities. 
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CHAPTER  TWO:  ENVIRONMENTAL  PLANNING 

OVERVIEW 

This  Chapter  defines  environmental  planning,  focussing  on  restoration  as  the  environmental 
investment  of  primary  interest  in  this  report.  Despite  its  focus,  all  the  principles,  tools  and  methods 
discussed  in  this  report  apply  to  virtually  all  types  of  environmental  planning. 

PLANNING 

Planning  is  defined  in  different  ways  by  different  people.  It  was  defined  recently  in  the  Corps’  Draft 
Flaming  Manual^  as  “the  deliberate  social  or  organizational  activity  of  developing  an  optimal  strategy  for 
solving  problems  and  achieving  a  desired  set  of  goals.”  Thus,  planning  is  goal-oriented  problem  solving. 
Defined  in  this  way,  planning  is  an  activity  that  is  not  confined  to  the  Corps’  planning  offices.  Personnel 
in  operations,  engineering  and  construction  are  doing  planning.  These  may  be  different  kinds  of  planning 
activities,  granted,  but  they  are  planning  activities  nonetheless. 

The  general  approach  to  planning  will  vary  with  the  people  doing  it  and  the  purposes  for  which  it 
is  done.  In  some  cases,  there  will  be  a  structured,  formal  approach  to  planning  as  there  is  in  traditional 
planning  studies.  In  other  cases,  the  problem  solving  is  more  gut-level  intuition  or  ad  hoc  in  nature.  In  order 
to  be  able  to  discuss  planning  we  need  a  common  understanding  of  what  we  mean.  Though  there  are  many 
models,  the  one  used  here  is  the  six-step  planning  process  defined  by  the  U.S.  Water  Resource  Council’s 
Economic  and  Environmental  Principles  and  Guidelines  for  Water  and  Related  Land  Resources 
Implementation  Studies,  commonly  called  the  “P&G”. 

The  six  planning  steps  are  summarized  in  Figure  1 .  Though  such  a  formal  framework  may  not  be 
used  in  all  the  environmental  planning  activities  that  are  undertaken  by  the  Corps,  they  easily  could  be.  It 
is  safe  to  assume  that  all  the  activities  should  at  least  have  the  fundamental  elements  of  identifying  problems, 
formulating  alternative  solutions  and 
identifying  the  best  solution  from  among  the 
possibilities.  Thus,  planning  as  used  in  this 
report  implies  a  more  or  less  formal  process 
comprising  the  six  steps. 

ENVIRONMENTAL  PLANNING 

Environmental  planning  is 
differentiated  from  the  more  traditional  types 
of  planning  the  Corps  has  done,  e.g.,  flood 
damage  reduction  and  navigation,  only  by  the 


*  The  report  is  circulating  in  draft  form  as  IWR  Report  95-R-I5,  December  1995, 
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specific  goals  to  be  achieved  and  the  problems  to  be  solved.  Though  some  consider  environmental  planning 
new,  it  is  not.  In  fact,  it  is  decades  old.  What  is  new  is  the  emphasis  it  is  receiving  and  the  ways  in  which 
it  is  evolving.  It  might  be  more  accurate  to  say  that  increasing  national  interest  in  and  importance  of 
environmental  values  is  receiving  new  emphasis  in  the  water  resources  planning  process,  rather  than  to 
suggest  that  '‘environmental  planning”  is  something  new. 

In  broad  terms,  environmental  planning 
could  be  considered  to  be  the  development  of 
optimal  strategies  that  preserve,  protect,  restore  or 
enhance  environmental  resources  and  systems  or 
that  mitigate  adverse  impacts  upon  environmental 
resources  and  systems.  Although  the  Corps  does 
not  have  the  authority  to  pursue  all  of  these 
strategies,  environmental  planning  within  the  Corps 
has  been  going  on  for  some  time. 

What  does  environmental  planning  mean  in 
the  context  of  the  Corps’  program?  Is  it  the  fish 
and  wildlife  work  the  Corps  has  been  authorized  to 
do  since  1958?  Does  it  mean  enhancing  hunting 
and  fishing  opportunities?  Does  environmental 
planning  mean  the  evaluation  of  environmental 
quality  (EQ)  effects  of  projects?  Environmental 
impact  assessment  is  nothing  new.  To  some, 
environmental  planning  involves  the  work  of  the 
Corps’  regulatory  program.  Clearly,  finding 
beneficial  uses  for  dredged  material  is  an  example 
of  environmental  planning  coincident  with  long¬ 
time  disposal  responsibilities.  Some  Corps 
personnel  suggest  hazardous,  toxic  and  radioactive 
waste  issues  encountered  by  the  Corps  require 
environmental  planning.  Others  think  of  ecosystem 
or  environmental  restoration  when  they  think  of 
environmental  planning. 

In  this  report,  environmental  planning 
means  all  these  things  and  more.  Any  Corps’ 
activities  that  involve  environmental  resources  and 
systems  directly  or  indirectly  can  be  considered  part 
of  environmental  planning.  Whether  this  work  is 
conducted  by  Planning,  Operations,  or  other  offices, 
the  work  is  essentially  planning  so  long  as  it  is 
involves  developing  optimal  strategies  for  goal-oriented  problem  solving. 


New  Emphasis  on  an  Old  Objective 

The  roots  of  the  Corps’  tradition  of 
environmental  planning  can  be  found  in  a  long  list  of 
authorization  acts,  environmental  laws  and  executive 
orders.  Examples  of  the  Corps’  new  emphasis  on 
environmental  planning  are  evident  in  a  number  of 
places.  These  include  the  Chief  of  Engineers’ 
February  1990  memorandum  on  “Strategic  Direction 
for  Environmental  Engineering”  and  the  Assistant 
Secretary  of  the  Army’s  June  1990  “Statement  on 
Environmental  Approaches”.  Policy  Guidance 
Letters  24  (March  1991)  and  35  (March  1992) 
followed  up  on  the  earlier  policy  pronouncements. 


New  emphasis  has  come  through  legislation 
as  well.  The  Water  Resources  Development  Act 
(WRDA)  of  1986  in  Section  1135  instructed  the 
Corps  to  consider  how  existing  projects  might  be 
altered  to  achieve  environmental  objectives.  Section 
1 103  of  this  act  authorized  the  Upper  Mississippi 
Environmental  Management  Plan.  Section  306  of 
WRDA  of  1990  authorizes  “environmental 
protection”  as  a  mission  for  the  Corps.  Section  307 
calls  for  a  wetlands  action  plan  with  a  goal  of  “no  net 
loss”  of  wetlands.  The  Coastal  Wetlands  Planning, 
Protection  and  Restoration  Act  of  1990  (Public  Law 
(PL)  101-646)  authorized  the  Corps  to  cooperate  with 
other  agencies  and  the  State  of  Louisiana  to  identify 
and  construct  wetlands  projects.  Section  204  of 
WRDA  of  1 992  encourages  the  Corps  to  find 
beneficial  uses  of  dredged  material.  Specific  actions 
like  the  1992  commitment  of  the  Corps  to  the 
Kissimmee  River,  Florida  restoration  project  have 
also  served  to  enhance  the  new  environmental 
emphasis. 
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THE  EEIRP 

This  report  is  being  developed  as  part  of  the  Evaluation  of  Environmental  Investments  Research 
Program  (EEIRP).  As  such,  it  has  a  narrower  focus  than  what  we  have  described  as  environmental  planning. 
The  focus  of  the  EEIRP,  hence  this  report,  is  on  the  evaluation  of  environmental  investments.  In  practical 
terms  this  has  come  to  mean  the  evaluation  of  environmental  restoration  projects.  This  would  include 
projects  authorized  by  Section  1135  of  the  Water  Resources  Development  Act  (WRDA)  of  1986  and  Section 
313  of  WRDA‘92  as  well  as  those  listed  in  the  sidebar  on  the  preceding  page. 

Although  this  report  tends  to  focus  more  narrowly  on  environmental  restoration  projects,  its  contents 
are  equally  applicable  to  any  area  of  environmental  planning.  For  example,  to  the  extent  that  the  uncertainty 
associated  with  habitat  evaluation  analyses  are  of  interest  to  the  restoration  of  degraded  ecosystem  structure 
function  and  dynamic  processes  (the  focus  of  this  report)  it  will  also  be  of  interest  for  mitigation  studies, 
environmental  impact  assessments,  and  any  other  endeavors  that  require  habitat  evaluations. 

Environmental  restoration  projects  are  new  and  they  are  different.  “Traditional”  Corps  planning  is 
national  economic  development  (NED)  oriented,  insofar  as  the  primary  objective  is  to  maximize  net  NED 
benefits.  Environmental  restoration  is  a  prime  example  of  “non-traditional”  planning,  i.e.,  planning  that  is 
not  driven  by  the  NED  objective.^  Absent  the  traditional  metric  of  NED  benefits  and  costs,  it  is  necessary 
to  develop  new  ways  to  evaluate  the  impacts  of  certain  kinds  of  projects.  One  of  the  purposes  of  the  EEIRP 
is  to  develop  some  of  these  evaluation  methods.  The  purpose  of  this  report  is  to  begin  to  consider  the  need 
for  and  ways  in  which  risk  and  uncertainty  analysis  may  be  introduced  to  the  evaluation  of  environmental 
infrastructure  aimed  primarily  at  restoring  degraded  ecosystems.  The  reason  for  taking  this  step  is  to 
improve  the  quality  of  decisions  made  in  these  planning  efforts. 

SUMMARY  AND  LOOK  FORWARD 

Environmental  planning  has  been  done  by  the  Corps  for  a  long  time.  The  growing  national  interest 
in  environmental  issues  has  created  a  new  emphasis  on  environmental  values  within  the  Corps’  program. 
This  new  emphasis  brings  with  it  new  initiatives.  Environmental  planning,  at  one  level,  is  nothing  more  than 
the  application  of  good  planning  techniques  to  environmental  problems  and  opportunities.  Although  this 
report  tends  to  focus  on  environmental  restoration  projects,  the  principles,  tools  and  methods  discussed 
throughout  are  equally  applicable  to  any  environmental  planning  activities. 

As  this  chapter  has  provided  an  overview  to  the  notion  of  environmental  planning,  so  the  next 
chapter  does  for  risk  and  uncertainty  analysis.  It  provides  the  reader  with  an  introduction  to  the  principle 
language  and  concepts  of  risk  and  uncertainty  analysis  that  will  be  used  in  subsequent  chapters  when 
environmental  planning  and  risk  analysis  are  brought  together. 


2 

These  non-traditional  planning  efforts  involve  environmental  infrastructure,  the  Corps’  regulatory  program,  support  for  the 
military  and  others,  master  planning,  regional  analysis,  drought  preparation  studies,  planning  assistance  to  the  States,  and  other  special  studies 
and  projects. 
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CHAPTER  THREE:  RISK  AND  UNCERTAINTY  ANALYSIS 

INTRODUCTION 

This  chapter  provides  the  reader  with  an  introduction  to  the  language  and  concepts  of  risk  and 
uncertainty  analysis  that  are  needed  in  order  to  introduce  risk  and  uncertainty  analysis  in  the  evaluation  of 
environmental  investments.  There  are  six  main  sections  including  this  one.  The  section  that  follows  is 
about  words.  It  defines  the  most  basic  terms,  considers  some  of  the  confusing  terminology  that  has  evolved 
around  the  concepts  of  risk  analysis  and  the  environment,  and  concludes  with  a  return  to  the  distinction 
between  what  is  risky  and  what  is  uncertain. 

The  third  section  of  this  chapter  presents  a  rationale  for  environmental  planners’  being  concerned 
with  risk  and  uncertainty  analysis.  The  bulk  of  the  remainder  of  the  chapter  comprises  two  sections  that 
provide  the  reader  with  a  taxonomy  of  uncertainty.  One  discusses  the  kinds  of  things  that  can  be  uncertain. 
The  other  discusses  the  sources  of  this  uncertainty.  Although  this  taxonomy  is  not  unique  to  environmental 
planning,  it  is  very  valuable  because  it  gives  an  expert,  untrained  in  uncertainty  analysis,  a  structured  way 
in  which  to  begin  to  think  about  uncertainty  in  his  or  her  domain  of  expertise.  The  chapter  concludes  with 
a  summary  and  a  look  forward  to  the  next  chapter. 

TERMINOLOGY 


BASIC  TERMS 

Uncertainty  describes  any  situation  in  which  we  are  not  absolutely  sure.  Risk  describes  a  situation 
in  which  there  is  a  chance  of  something  bad  happening.  Risk  analysis  is  the  analysis  of  risky  situations.  Risk 
analysis  is  broken  into  two  parts:  risk  evaluation  and  risk  management.  Risk  evaluation  is  the  objective 
analysis  of  the  situation  that  answers  the  question,  “How  risky  is  this  situation?”  Risk  management  then 
answers  the  question,  “What  shall  we  do  about  it?” 

Risk  and  uncertainty  analysis  is  not  a  section  of  a  report  that  is  added  after  the  study  is  complete. 
It  is  not  playing  “what  if’  games  with  a  critical  variable  or  two  in  order  to  satisfy  a  requirement.  Introducing 
risk  and  uncertainty  analysis  to  environmental  restoration  planning  means  changing  the  way  analysts 
approach  the  planning  process.  At  its  most  basic  level,  it  requires  planners  to  recognize  and  admit  there  is 
really  very  little  of  which  they  are  absolutely  sure.  This  humbling  admission  can  then  lead  planners  to 
approach  their  goal-oriented  problem  solving  tasks  in  a  more  informed  fashion,  because  recognizing  the 
limits  of  our  expertise  and  knowledge  is  always  one  of  wisdom’s  more  difficult  tasks.  Never  is  this  more 
the  case  than  in  addressing  complex  environmental  systems. 

THE  SPECIALIZED  TERMINOLOGY  OF  RISK  AND  UNCERTAINTY 

The  previous  section  offers  some  simple  definitions  of  basic  terms  used  in  risk  and  uncertainty 
analysis.  Despite  that  apparent  simplicity,  the  language  of  risk  and  uncertainty  analysis  can  be  very 
confusing  because  there  are  many  disciplines  that  make  use  of  its  tools  and  methods.  Unfortunately,  each 
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specialty  application  has  developed  its  own  language.  For  example,  what  one  group  calls  risk  analysis 
another  calls  risk  assessment.  These  discrepancies  can’t  be  cleared  up  here  but  the  meaning  of  terms  as  used 
in  this  report  can  be  clarified.  If  you  read  the  risk  and  uncertainty  literature,  expect  to  encounter  a  rather 
confusing  terminology  that  will  not  always  match  the  simple  definitions  offered  here.  Following  are 
overviews  of  the  more  common  specialized  terminology  of  risk  and  uncertainty  analysis. 

The  insurance  industry  first  coined  the  term  risk  management  to  describe  the  tools  and  methods 
employed  by  businesses  to  limit  and  control  the  economic  risks  they  faced.  Loss  through  accident,  fire, 
disaster,  and  so  on  were  among  the  first  hazards  businesses  faced.  In  recent  decades,  the  hazards  to  a  firm’s 
finaneial  health  have  mutated  to  the  legal,  political,  and  financial  realms  as  well.  Risk  management,  in  this 
context,  is  the  identifieation,  measurement,  and  treatment  of  exposures  to  potential  accidental  losses.  Risk 
management  is  essential  to  the  survival  and  profitability  of  a  business.  Risk  management  is  generally  not 
what  we  mean  when  we  speak  about  risk  and  uncertainty  analysis  for  the  Corps  of  Engineers. 

Environmental  risk  analysis  (a.k.a.,  environmental  risk  assessment  or  environmental  health  risk) 
sounds  exactly  like  what  this  report  is  talking  about.  It  is  not.  The  term  “environmental  risk  analysis”  has 
most  commonly  come  to  mean  the  assessment  of  toxic  risk  to  a  species.  Most  often,  the  toxic  risk  is 
exposure  to  some  chemical,  radioactive  or  other  hazardous  material  and  the  species  of  most  interest  is,  of 
course,  humans.  The  U.S.  Environmental  Protection  Agency  (EPA)  has  developed  principles  of 
environmental  risk  analysis  that  include  the  following  four  steps. 

First,  hazard  identification  is  the  identification  of  the  adverse  effects  a  substance/agent  can  cause. 
Second,  the  dose-response  assessment  is  the  process  of  establishing  or  characterizing  the  relationship 
between  the  dose  of  an  agent  and  the  incidence  of  adverse  health  effects.  The  Corps  uses  this  concept 
explicitly  in  the  bioassays  it  conducts  when  identifying  contaminants  in  dredged  material.  LD50,  or  lethal 
dose  50,  is  that  level  of  contaminant  (i.e.,  the  dose)  that  kills  50%  of  the  population  (i.e.,  the  response).  It 
uses  the  concept  implicitly  in  its  expected  annual  damage  computations  for  flood  damage  reduction  studies. 
The  dose  is  flood  stage,  the  response  is  damage.  Third,  exposure  assessment  is  the  process  of  estimating  the 
intensity,  frequency,  and  duration  of  human  or  animal  exposure  to  an  agent  in  the  environment.  Fourth,  risk 
characterization  is  the  final  step.  It  is  the  synthesis  of  the  dose-response  and  exposure  assessments,  often 
presented  in  the  form  of  a  toxicity/exposure  or  T/E  ratio.  The  purpose  of  this  step  is  to  estimate  the  incidence 
of  health  effects  under  various  conditions  of  human  or  animal  exposure  described  in  the  exposure 
assessment.  Although  the  Corps  makes  both  explicit  and  implicit  use  of  these  coneepts,  environmental  risk 
analysis  is,  generally,  not  what  we  meant  by  risk  and  uncertainty  analysis  in  this  report. 

Human  health  risk  assessment  is  split  into  cancer  risks  and  non-cancer  risks.  As  environmental  risk 
analysis  has  more  and  more  come  to  be  identified  with  risks  to  human  health  through  exposure  to  chemicals 
and  other  agents  in  the  environment,  it  has  been  necessary  to  develop  new  terminology.  Ecological  risk 
assessment  is  one  of  these  terms.  Rather  than  concentrating  on  the  effects  of  a  toxic  substance  on  a  single 
species,  ecological  risk  assessments  are  concerned  with  the  effect  of  stressors  found  in  the  environment  that 
can  affect  single  species,  communities  of  species,  and  entire  ecosystems. 
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The  risk  assessment  paradigm  under  development  for  ecological  risk  assessment  is  essentially  an 
adaptation  of  the  four-step  environmental  risk  analysis  paradigm  summarized  above.  Instead  of  a  dose- 
response  relationship,  a  stress-response  assessment  has  been  proposed.  Ecosystems  are  subject  to  a  much 
wider  range  of  stresses  including  man-made  and  natural,  chemical  and  non-chemical.  Generally,  ecosystem 
risk  assessment  is  not  what  is  meant  by  risk  and  uncertainty  in  the  Corps’  context.  However,  the  paradigm 
under  development  holds  considerable  promise  for  use  in  analyses  related  to  habitat  loss,  global  climate 
change  and  other  non-chemical  ecological  stresses.  Once  basic  acceptance  of  risk  and  uncertainty  concepts 
has  been  established,  adaptation,  if  not  adoption,  of  some  or  all  of  the  ecological  risk  assessment  paradigm 
under  development  may  warrant  serious  consideration. 

An  early  effort^  at  assessing  the  extent  to  which  risk  and  uncertainty  analysis  is  currently  used  by 
environmental  planners  indicated  that  Corps  personnel  have  a  rather  limited  view  of  what  risk  and 
uncertainty  means.  The  predominant  notion  of  risk  analysis  is  that  it  applies  when  there  is  some  danger  of 
loss  of  human  life  or  failure  of  a  project.  There  is  no  apparent  awareness  of  uncertainty  analysis.  Though 
loss  of  life  and  project  failure  are  part  of  risk  and  uncertainty  analysis,  they  are  only  a  small  part. 

To  distinguish  the  Corps’  applications  of  risk  and  uncertainty  analysis  from  those  addressed  above, 
the  types  of  analyses  of  interest  to  the  Corps  will  be  called  risk  and  uncertainty  in  water  resources.  Thus, 
we’re  speaking  about  risk  and  uncertainty  analysis  in  a  generic  sense  with  specific  applications  of  the  general 
theory  and  tools  to  situations  that  involve  water  resources. 

For  the  most  part.  Corps  projects  are  concerned  with  a  wide  variety  of  nontoxic  risks  that  are  site- 
specific  and  are  caused  by  such  processes  as  dredging  a  navigation  channel,  construction  of  a  dam,  creation 
of  wetlands,  or  natural  disaster  recovery  efforts.  The  impacts  tend  to  be  systematic  rather  than  incremental. 
That  is,  a  single  event  -  like  the  construction  of  a  project,  a  flood,  a  volcanic  eruption,  an  earthquake  - 
produces  widespread  repercussions.  These  could  include  modification  of  the  hydrologic  cycle,  alteration 
of  natural  hydraulics,  or  changes  in  fish  and  wildlife  habitat;  i.e.,  temporary  or  permanent  changes  in  natural 
and  human  systems.  The  damages  that  result  from  these  changes  may  be  economic,  environmental  or  safety 
related.  Incremental  effects  are  in  some  sense  additive,  like  the  build-up  of  toxins  in  the  food  chain  or 
erosion  of  a  streambank  or  island. 

A  paradigm  is  an  example  intended  to  demonstrate  how  something  is  to  be  done.  It  provides  a 
pattern  that  can  be  followed  by  others.  The  EPA  has  devoted  considerable  effort  to  developing  risk  analysis 
paradigms.  The  four-step  process  described  above  is  an  example  of  a  paradigm.  The  Corps  does  not  rely 
on  a  single  paradigm. 

EC  1 105-2-205,  Risk  Analysis  Framework  for  Evaluation  of  Hydrology/Hydraulics  and  Economics 
in  Flood  Damage  Reduction  Studies,  presents  a  paradigm  for  calculating  expected  annual  damages.  Risk- 
based  models  have  been  developed  for  use  in  major  rehabilitation  projects  and  dredging.  Underlying  all 
these  risk-based  models  is  a  basic  theoretical  framework  comprising  the  three  steps  shown  in  Figure  2.  In 


3 

“Compilation  and  Review  of  Completed  Restoration  and  Mitigation  Studies  in  Developing  an  Evaluation  Framework  for 
Environmental  Resources,  Volumes  I  and  H”,  IWR  Reports  95-R-4  and  95-R-5,  April  1995. 
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each,  the  hazard  (flooding,  component  failure,  uncertain  quantities)  are  identified.  The  probability  of  the 
hazard  causing  a  problem  is  quantified  and  the  consequence  of  the  problem  caused  by  the  hazard  is 
quantified  and  modeled. 

The  EPA  health  risk  paradigm  can  be 
adapted  for  use  by  the  Corps.  Flooding  is  a  hazard 
that  is  readily  amenable  to  the  hazard,  dose- 
response,  exposure,  and  risk  characterization 
model.  Contaminant  analysis  of  dredged  material 
requires  no  adaptation.  There  are  easily  imagined 
potential  uses  for  an  ecological  risk  assessment 
paradigm.  But  the  Corps’  work  will  never  be 
reduced  to  a  single  or  even  a  few  risk  and 
uncertainty  analysis  paradigms.  The  work  is  too  diverse  and  too  unique  in  its  hazards  and  their  consequences 
to  ever  reduce  to  one  or  two  established  ways  of  approaching  risk  and  uncertainty.  Nonetheless,  there  are 
recurring  and  important  situations  that  do  lend  themselves  to  the  development  of  paradigms  and  models  that 
could  be  used  over  and  over  throughout  the  organization.  In  time,  some  may  be  developed  for  the  Corps’ 
environmental  investment  work.  This  report  is  content  to  speculate  about  where  these  paradigms  might  arise 
and  where  they  are  never  likely  to  occur. 

RISK  OR  UNCERTAINTY 

The  simple  definitions  offered  above  may  not  be  enough  to  help  the  new  reader  to  distinguish 
between  risk  and  uncertainty.  It  is  less  important  that  a  situation  is  properly  classified  as  risk  or  uncertainty 
then  that  uncertain  situations  that  are  important  to  the  decision  process  are  recognized  and  dealt  with 
forthrightly  and  as  effectively  as  possible  throughout  the  planning  process.  Any  situation  in  which  the 
outcome  or  final  result  is  not  known  with  complete  certainty  is  an  uncertain  one.  This  covers  a  lot  of  ground. 
We  are  certain  c-a-t  spells  cat.  We  are  certain  of  our  names.  We  are  certain  the  sun  will  rise  again 
tomorrow,  aren’t  we?  We  are  sure  we  set  our  keys  here  a  minute  ago.  We  know  it’s  going  to  rain  tomorrow. 
When  you  get  right  down  to  it,  there  is  relatively  little  in  our  lives  that  we  can  be  certain  about. 

Not  all  the  uncertainty  we  face  is  equally  important.  For  all  intents  and  purposes,  we  can  ignore 
uncertainty  about  the  sun’s  rising  and  setting.  Though  not  absolutely  certain,  many  things  are  effectively 
certain.  Much  of  our  scientific  theory  falls  in  this  category  of  knowledge.  Other  things,  like  where  our  keys, 
glasses,  umbrellas,  and  the  like  currently  reside  are  rather  trivial  in  the  scheme  of  things  and  can  be  ignored. 

Some  uncertain  things  fall  in  between  these  extremes  and  they  need  to  be  addressed.  You  find 
yourself  in  a  strange  town  at  night,  is  it  safe  to  walk  in  this  neighborhood?  You  don’t  understand  the 
question  on  a  final  exam.  Do  you  drink  the  water  in  a  foreign  country?  You  have  no  idea  where  Faraway 
Street  is. 

It’s  the  same  in  water  resources  planning  generally  and  environmental  restoration  planning 
specifically.  There  are  many  things  that  are  uncertain  and  many  sources  of  uncertainty  (topics  covered  later 


Figure  2:  Elements  of  Risk  Analysis 

•  Hazard  identification 

•  Probability  estimation 

•  Consequence  modeling 
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in  this  chapter).  Not  all  are  created  equal.  The  first  step  in  dealing  with  this  uncertainty  in  a  rational  way 
is  going  to  be  to  figure  out  what  uncertainty  is  most  important  and  what  uncertainty  can  be  ignored.  The 
basic  criteria  for  establishing  that  importance  is  to  consider  whether  the  uncertain  variable  could  have  a 
significant  effect  on  the  decision  to  be  made.  If  it  could,  it’s  important.  If  it’s  important,  you  need  to 
consider  the  uncertainty.  Some  tools  and  methods  that  can  be  used  to  address  various  uncertain  situations 
are  presented  in  the  next  chapter. 

There  is  a  class  of  uncertain  situations  that  arise  frequently  enough  to  earn  them  the  special 
designation  of  risky  situations,  as  shown  in  Figure  3.  Risk  was  defined  above  as  the  chance  of  something 
bad  happening.  The  chance  is  often  unknown  as  are  the  nature  and  extent  of  the  “bad”  thing  that  might 
happen.  Risky  situations  are,  then,  characterized  by  uncertain  likelihood  of  occurrence  and  uncertain 
consequences;  clearly,  a  case  of  uncertainty. 


Figure  3:  Risk  &  Uncertainty  Relationship  1 


A  few  examples  help  illustrate  these  concepts.  We  may  be  uncertain  about  how  an  ecosystem 
functions;  we  may  be  uncertain  about  a  habitat  suitability  index;  we  may  not  be  comfortable  extrapolating 
from  the  LD50  to  the  No  Observed  Adverse  Effect  Level  (NOAEL)  of  a  contaminant;  and,  we  may  be 
uncertain  about  the  true  output  of  a  mitigation  or  restoration  project.  On  the  other  hand,  there  is  a  risk  of 
drought  or  flood  and  there  is  a  risk  that  our  project  will  not  perform  as  expected.  The  differences  can  be 
subtle  at  times.  How  we  label  a  decision  problem  is  less  important  than  that  we  entertain  it.  The  terms,  risk 
analysis,  risk  and  uncertainty  analysis,  and  uncertainty  analysis  are  used  as  virtual  synonyms  in  this  report, 
less  because  there  is  no  difference  than  because  the  difference  is  relatively  unimportant. 

One  final  note  on  jargon  is  necessary.  Risk-based  analysis  is  a  term  invented  by  the  Corps  that  is 
essentially  a  synonym  for  risk  and  uncertainty  analysis.  Risk-based  analysis  is  defined  in  EC  1 105-2-205 
as; 
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“...an  approach  to  evaluation  and  decision-making  that  explicitly,  and  to  the  extent  practical,  analytically 
incorporates  considerations  of  risk  and  uncertainty,” 

WHY  BOTHER  WITH  RISK  AND  UNCERTAINTY? 

Each  time  risk  and  uncertainty  analysis  has  been  introduced  to  a  new  area  of  the  Corps’  program  the 
reaction  has  always  been  the  same.  What  is  it?  Is  it  just  one  more  thing  we  have  to  do?  Is  it  another  way 
for  people  to  keep  us  from  doing  what  needs  to  be  done.  How  do  you  do  this?  Who’s  going  to  do  this? 
What’s  the  point? 

Four  reasons  are  offered  for  Corps  personnel,  involved  in  environmental  activities,  to  be  concerned 
with  risk  and  uncertainty  analysis.  First,  planning  requires  decision-making  under  uncertainty,  so  it’s  just 
good  planning.  Second,  risk  and  uncertainty  analyses  are  already  being  used  to  improve  decision-  making 
in  other  parts  of  the  Corps’  program  without  difficulty.  Third,  there  is  a  growing  demand  for  it.  Fourth, 
there  is  growing  political  interest  in  providing  regulatory  reform  that  focuses  national  resources  on  the 
greatest  risks  to  human  health,  safety,  and  the  environment.  Any  one  of  these  reasons  should  be  sufficient 
to  encourage  environmental  resources  planners  and  managers  to  use  risk  and  uncertainty  analysis.  Taken 
together  they  form  a  rationale  for  beginning  to  incorporate  risk  and  uncertainty  analyses  into  the  Corps 
environmental  planning. 

The  first  reason,  that  using  risk  and  uncertainty  analysis  is  just  good  planning,  may  be  the  most 
compelling.  If  planning  is  an  attempt  to  control  future  consequences  through  present  actions,  an  appropriate 
view  of  ecosystem  restoration  planning,  it  is  obvious  we  are  confronted  with  considerable  uncertainty.  We 
don’t  fully  understand  how  complex  ecosystems  function.  We  have  to  work  without  all  the  data  we’d  like 
to  have.  We’re  not  sure  how  our  alternatives  will  perform  and  often  don’t  know  exactly  what  they  will  cost 
to  build.  There  is  also  considerable  uncertainty  over  how  alternatives  can  best  be  evaluated. 

Faced  with  these  and  other  sources  of  uncertainty  it  would  be  unprofessional  and  irresponsible  not 
to  admit  and  face  the  uncertainty  with  which  we  are  confronted.  Given  the  reality  of  uncertainty,  it  only 
makes  sense  to  deal  with  it  in  a  rational  and  systematic  way.  This  requires  the  application  of  the  tools  and 
methods  of  risk  and  uncertainty  analysis.  When  a  problem  exists,  it  makes  sense  to  recognize  it  and  deal 
with  it.  That  is  simply  good  planning.  Good  planning  leads  to  good  plans  and  good  decisions. 

The  second  reason  for  using  risk  and  uncertainty  analysis  is  that  it  is  already  being  successfully  used 
by  the  Corps.  Risk  and  uncertainty  are  nothing  new.  Applying  these  concepts  to  environmental  planning 
may  be  new,  but  it’s  nothing  more  than  the  logical  extension  of  good  analysis  to  a  new  subject  area.  Risk 
analysis  began  with  the  dam  safety  program  more  than  a  decade  ago.  Flood  damage  reduction  studies  are 
required  to  use  a  risk-based  analysis  to  evaluate  hydrology/hydraulics  and  economics  (see  EC  1 1 05-2-205). 
Major  rehabilitation  of  locks  and  dams  has  been  relying  on  risk-based  analysis  for  years  and  the  tools  and 
methods  in  use  have  grown  increasingly  sophisticated.  Dredging  decisions  are  now  assisted  by  risk-based 
analytical  tools.  Dredged  material  disposal  programs  that  use  the  Evaluation  of  Dredged  Material  Proposed 
for  Ocean  Disposal  -  Testing  Manual,  1991,  also  known  as  the  “Green  Book”,  have  been  applying  risk-based 
analytical  techniques  for  years.  Risk  and  uncertainty  analyses  have  been  used  for  channel  width 
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determinations  for  navigation  projects.  Pilot  studies  have  been  conducted  to  explore  the  use  of  risk  and 
uncertainty  tools  and  methods  in  determining  budgeting  priorities  of  operation  and  maintenance  budgets. 
These  techniques  are  examples  of  some  of  the  applications  that  have  been  used  to  improve  the  quality  of 
decision-making  within  the  Corps  of  Engineers. 

Since  the  inception  of  the  Corps’  Risk  Analysis  Research  Program  at  the  Institute  for  Water 
Resources  (IWR),  there  has  been  a  snowball  effect  on  the  application  of  risk  and  uncertainty  analysis 
methods  to  water  resources  planning.  Now  that  the  feasibility  and  value  of  these  methods  have  been  proven, 
efforts  to  spread  them  to  new  applications  within  the  Corps  program  continue.  Incorporating  risk  and 
uncertainty  analysis  into  environmental  investment  programs  is  just  part  of  the  natural  evolution  of 
improving  decision-making  processes.  It  just  makes  sense. 

The  third  reason  is  that  there  is  a  growing  demand  for  risk-based  analysis.  Growing  public 
awareness  of  environmental  issues  and  of  risks  in  general  has  raised  public  expectations  of  the  quality  of 
environmental  risk  analysis.  Informed  public  debate  has  resulted  in  an  unwillingness  of  public  officials  and 
the  public  to  leave  the  discussion  of  environmental  risks  to  the  so-called  experts.  “Suppose,”  the  non-Federal 
partner  to  a  marsh  creation  project  asks,  “we  put  up  our  money  and  the  project  doesn’t  work,  will  it  be 
redone?  Who  pays  for  that?” 

This  growing  demand  for  risk  analysis  leads  directly  to  a  fourth  argument  for  incorporating  risk  and 
uncertainty  into  environmental  project  planning;  the  current  Administration  and  Congress  have  begun  to 
move  in  the  direction  of  requiring  it.  Any  teenager  recognizes  the  compelling  nature  of  the  “because  I  said 
so”  rationale  for  doing  something.  There  is  a  good  chance  that  that  may  soon  become  a  compelling  reason 
for  Corps  personnel  to  conduct  risk  and  uncertainty  analysis. 

On  August  2,  1994  the  Clinton  Administration  published  its  “Draft  Administration  Risk  Assessment 
Principles”.  A  review  of  these  principles  for  the  Chief  of  Engineers  by  IWR  said  in  part: 

“Without  any  reservations,  the  principles  are  sound  and  are  virtually  the  same  as  those  that 

the  Corps  has  been  promoting  in  its  various  planning  functions.. 

In  addition  to  these  principles,  numerous  bills  were  recently  introduced  in  Congress  to  further  the 
cause  of  risk  and  uncertainty  analysis.  Two  examples  were  House  Resolution  (H.R.)  1022  the  “Risk 
Assessment  and  Cost-Benefit  Act  of  1995”  and  HR  690  the  “Risk  Assessment  and  Cost-Benefit  Analysis 
Act  of  1995.’”  Either  would  have  mandated  the  conduct  of  risk  analyses  that  would  have  affected  the  Corps 
of  Engineers.  By  beginning  to  incorporate  risk  and  uncertainty  analysis  into  environmental  planning  now, 
planning  and  decision-making  can  be  improved  and  Corps  personnel  will  be  that  much  further  along  the 
learning  curve  if  and  when  risk  analysis  becomes  a  Federal  requirement. 


Memorandum  for  Chief,  CECW-AO/EINARSEN  from  CEWRC-IWR-P  dated  16  Nov  94,  Subject:  Draft  Administration 
“Principles  for  Risk  Assessment,  Management  and  Communication.” 

^  Senate  Bills  229  and  333  and  House  Resolution  96  are  additional  risk  analysis  items  before  the  104th  Congress. 
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If  your  goal  as  a  reader  is  to  understand  the  basic  notions  of  risk  and  uncertainty,  stop  here.  The 
remainder  of  this  chapter  goes  beyond  that  modest  goal  to  offer  a  few  ideas  that  might  help  the  new  reader 
to  think  about  uncertainty  in  a  systematic  manner.  It  begins  with  a  classification  system  or  taxonomy  of  the 
types  of  things  that  can  be  uncertain.  From  there  it  proceeds  to  a  taxonomy  of  the  sources  of  uncertainty. 
Much  of  the  material  in  the  remainder  of  this  chapter  is  taken  from  Uncertainty,  a  Guide  to  Dealing  with 
Uncertainty  in  Quantitative  Risk  and  Policy  Analysis  by  M.  Granger  Morgan  and  Max  Henrion. 

TYPES  OF  UNCERTAINTY 

How  can  we  break  the  “uncertain”  world  down  into  just  a  few  components  so  we  can  think  rationally 
about  them?  What  kinds  of  things  are  uncertain?  Planners  can  think  of  uncertainty^  as  coming  in  three 
classes:  uncertainty  in  knowledge;  uncertainty  about  quantities;  and  uncertainty  about  models.  Quarrel  with 
this  if  you  will,  it’s  a  purely  arbitrary  way  to  break  things  down.  It  does,  however,  provide  a  way  to  think 
about  uncertainty  encountered  in  environmental  restoration  planning.  By  thinking  about  the  knowledge 
required  to  approach  a  problem,  the  data  needed  to  analyze  it  and  the  models  used  in  the  analysis,  one  has 
a  structured  way  to  begin  to  think  about  the  uncertainty  inherent  in  any  situation. 

In  the  sub-sections  that  follow  each  of  these  three  classifications  are  addressed.  The  discussion  turns 
to  the  sources  of  uncertainty  in  the  next  section. 

KNOWLEDGE  UNCERTAINTY 

Some  things  are  uncertain  for  the  most  basic  of  reasons  -  we  just  don’t  know.  That  lack  of 
knowledge  can  arise  from  two  different  sources.  You  may  not  know  the  capital  of  Belgium.  That  is  an 
uncertainty  that  arises  from  ignorance.  The  information  is  known  (Brussels),  but  not  by  you.  The  cure  is 
relatively  obvious,  obtain  the  knowledge.  This  can  be  done  through  education,  training,  talking  to  experts, 
or  acquiring  experts  through  hiring,  contract  or  coordination. 

In  other  cases,  some  things  are  unknown  at  this  point  in  time  but  they  may  become  known  in  the 
future.  The  problem  may  be  incomplete  theory;  we  might  simply  lack  a  framework  for  thinking  about  or 
understanding  a  situation.  Even  when  the  theory  is  complete  a  lack  of  data  may  limit  our  understanding  of 
a  situation.  Examples  of  this  kind  of  uncertainty  include  cures  for  various  diseases;  knowledge  of  ungaged 
stream  flow  regimes;  and  the  performance  of  environmental  projects.  In  time,  cures  may  be  discovered,  flow 
data  collected  and  many  projects  monitored.  As  this  happens  uncertainty  is  reduced.  The  solutions  here  are 
more  research,  more  time,  more  inspiration  and  more  perspiration. 

There  are  other  things  that  are  fundamentally  unknowable.  These  include  the  questions  of  the  great 
philosophers  like,  “Can  God  make  a  rock  so  heavy  He  can’t  lift  it?”  and  “What  makes  you  so  sure  God  is 
a  He?”  It  includes  many  more  serious  value-based  questions  about  what  “should”  be  done  and  the  “right” 
thing  to  do  in  various  situations.  It  also  includes  things  that  are  random  and  unpredictable,  like  the  year  in 


Here  and  in  the  remainder  of  this  chapter  we  use  uncertainty  in  the  inclusive  sense  that  includes  risky  situations  as  well. 
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which  the  standard  project  flood  will  occur.  Many  of  the  risk  situations  fall  into  this  category.  There  is  no 
cure  for  this  kind  of  uncertainty.  Situations  like  these  must  often  be  represented  with  probabilities. 

One  of  the  most  common  sources  of  uncertainty  is  our  limited  knowledge.  Sometimes  it  is  caused 
by  a  lack  of  expertise,  data,  or  understanding.  Other  times  it’s  caused  by  the  fundamental  randomness  of 
the  universe.  Whatever  it’s  cause,  it’s  a  problem  that  may  need  to  be  addressed. 

QUANTITY  UNCERTAINTY 

Planning  requires  information,  lots  of  it.  That  information  is  frequently  gathered,  organized, 
summarized,  analyzed  and  expressed  in  a  quantitative  fashion.  There  are  167  acres  of  emerging  wetlands. 
There  are  six  eagle  nests.  The  herbaceous  cover  suitability  index  is  0.6.  The  cost  of  a  duck  box  is  $320. 
The  quantities  used  in  the  planning  process  are  frequently  a  major  source  of  uncertainty. 

Table  1  presents  a  classification  of  empirical  quantities  encountered  in  planning  as  well  as  some 
methods  for  dealing  with  the  uncertainty  that  may  be  found  in  them.  Empirical  quantities  are  measurable 
properties  of  the  real  world  systems  being  analyzed  and  modeled.  Each  type  of  quantity  is  discussed  briefly 
below. 


Table  1:  Types  of  Empirical  Quantities  Encountered  in  the  Planning  Framework 

Type  of  Quantity 

Example 

Treatment  of  Uncertainty 

Empirical  quantities 

Mortality  rate,  stream  discharge 

Probabilistic,  parametric 

Defined  constant 

Gallons  per  acre-foot 

Certain  by  definition 

Decision  variable 

Level  of  protection,  mitigation  goal 

Parametric 

Value  parameter 

Discount  rate,  value  of  life 

Parametric 

Index  variable 

Acreage,  longitude  &  latitude 

Certain  by  definition 

Model  domain  parameter 

Study  area,  planning  horizon,  base  year 

Parametric 

Outcome  criterion 

BCR,  incremental  costs,  habitat  units 

Depends  on  treatment  of  inputs 

Empirical  Quantities 

Empirical  quantities  are  the  most  common  quantities  encountered  in  a  study.  To  be  empirical, 
variables  must  be  measurable  in  principle.  Empirical  quantities  are  the  measurable  properties  of  the 
resources  and  factors  that  need  to  be  measured  in  the  planning  process.  They  include  quantities  such  as 
population  counts  for  significant  species,  water  temperatures,  stream  flows,  quantity  estimates,  the  percent 
of  unsubmerged  substrate,  the  probability  that  a  species  will  become  locally  extinct  in  the  next  ten  years. 
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average  erosion  or  shoaling  rates,  roughness  coefficients,  contaminant  loads,  and  so  on.  Empirical  quantities 
need  not  be  numerical.  They  can  be  nominal  as  well,  like  the  lists  of  species  present  in  a  study  area. 

To  illustrate  the  basic  idea,  suppose  weVe  trying  to  estimate  the  resident  population  of  white  tail 
deer.  There  is  a  single  true  value  for  the  white  tail  deer  population  at  a  given  place  and  point  in  time. 
Whether  we  can  actually  determine  that  true  value  is  another  matter;  we  may  have  census  data  or  only 
sample  information.  In  either  case  there  is  a  strong  likelihood  we  do  not  know  the  true  population  with 
certainty. 

Defined  Constant 

One  type  of  quantity  is  always  certain.  These  are  defined  constants.  There  will  always  be  43,560 
square  feet  to  an  acre  and  there  are  326,000  gallons  of  water  in  an  acre-foot.  These  values  are  certain  by 
definition  and  we  never  have  to  worry  about  their  uncertainty. 

Decision  Variables 

Decision  variables  are  quantities  the  planning  team  exerts  direct  control  over.  They  are  sometimes 
called  control  variables  or  policy  variables.  There  may  be  uncertainty  about  how  densely  to  plant  marsh 
grasses  when  trying  to  establish  tidal  wetlands  or  the  best  level  of  environmental  mitigation.  It  makes  no 
sense,  however,  to  be  uncertain  about  the  “true”  value  of  grass  density  or  environmental  mitigation  because 
unlike  empirical  quantities,  decision  variables  have  no  true  values.  Their  values  are  selected  by  the  planning 
team.  Representing  them  as  probabilistic  variables  detaches  them  from  the  decision  process.  It  is  better  to 
vary  decision  variables  systematically  using  sensitivity  analysis,  also  known  as  parametric  variation. 

Value  Parameters 

Value  parameters  represent  the  preferences  of  the  planning  team  or  the  people  they  represent  -  the 
general  public,  resource  agencies,  higher  authority.  The  federal  discount  rate  is  an  example  of  a  value 
parameter.  The  choice  of  this  uncertain  variable  has  been  decided  for  the  Corps  by  Congress.  At  times, 
decision-makers  assign  weights  to  various  project  results  to  facilitate  trade-off  analysis.  These  weights  may 
be  explicit  or  implicit  but  they  reflect  the  values  of  the  decision-maker  and  should  not  be  varied  randomly. 

It  is  not  unusual  for  a  value  parameter  to  be  treated  as  an  empirical  quantity  and  represented  by  a 
probability  distribution.  This,  however,  is  usually  a  mistake,  sometimes  a  serious  one.  Value  parameters 
are  value  judgments.  If  the  decision-maker  is  not  sure  what  she  prefers  it  is  not  helpful  to  leave  the  value 
judgment  to  chance.  It  is  better  to  vary  the  values  parametrically,  i.e,  first  try  one,  then  another.  This  allows 
the  team  to  learn  how  changes  in  the  parameter's  value  affect  the  outcomes  in  which  they  are  interested. 

People  are  often  least  sure  about  value  parameters  and  they  may  contribute  significantly  to  the 
uncertainty  about  the  best  alternative  plan.  If  these  quantities  are  treated  as  probabilistic  it  may  mask  the 
effect  of  the  different  value  choices  on  the  planning  objectives.  A  systematic  sensitivity  analysis  can  be  far 
more  illuminating  and  can  help  decision-makers  better  understand  and  refine  their  preferences. 
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Index  Variables 

Index  variables  are  used  to  identify  and  locate  information  in  a  model.  For  example,  the  base  year, 
project  year  10,  grid  cell  230,  90o50’  longitude  and  30o42'  latitude,  are  all  index  variables.  There  is  no 
reason  to  be  uncertain  about  the  value  of  an  index  variable.  However,  as  long  as  humans  are  doing  the  work, 
human  errors  in  measurement  are  always  possible. 

Model  Domain  Parameters 

Model  domain  parameters  specify  the  scope  of  the  study.  We  identify  the  Potomac  watershed  as  the 
study  area  and  a  50-year  planning  horizon.  Values  like  these  define  the  extent  of  the  study.  The  planning 
horizon  delimits  the  temporal  domain  of  the  study,  the  study  area  delimits  the  spatial  domain.  Considering 
levee  heights  in  two-foot  increments  and  defined  project  increments  in  the  incremental  cost  analysis  are 
additional  examples  of  model  domain  parameters. 

There  are  no  true  model  domain  parameter  values  and  it  would  be  inappropriate  to  represent  these 
model  domain  parameters  with  probability  distributions.  The  choice  of  the  domain  values  is  up  to  the 
planning  team. 

Outcome  Criteria 

The  outcome  criteria  are  the  variables  we  use  to  rank  or  measure  the  desirability  of  possible 
outcomes.  The  Corps  has  traditionally  used  net  National  Economic  Development  (NED)  benefits  in  its 
planning  studies.  Habitat  suitability  indices,  habitat  units  and  incremental  costs  are  other  outcome  criteria 
in  common  usage  for  mitigation  and  restoration  planning. 

New  decision  criteria  will  evolve  as  this  branch  of  planning  develops.  These  outcome  criteria  will 
be  deterministic  if  there  is  no  uncertainty  in  the  inputs  used  to  develop  them  and  there  is  an  unambiguous 
analytical  method  to  develop  the  criteria.  However,  if  the  inputs  are  uncertain  or  there  are  no  analytical 
methods  for  estimating  the  criteria,  the  outcome  criteria  will  be  uncertain.  For  example,  if  the  habitat 
suitability  index  is  uncertain  and  the  acreage  is  certain,  habitat  units  will  be  an  uncertain  outcome  criteria 
because  it  is  the  product  of  an  uncertain  variable  times  a  certain  constant. 

MODEL  UNCERTAINTY 

Models  are  simplifications  of  reality  used  in  order  to  understand  or  represent  complex  situations  and 
systems.  Models  can  be  physical  or  mathematical,  the  latter  being  far  more  common.  The  ideal  model  is 
a  simple  one  that  closely  replicates  the  functioning  of  a  system.  Model  uncertainty  is  a  type  of  uncertainty 
that  can  be  more  important  than  knowledge  or  quantity  uncertainty  in  some  cases. 

The  issue  with  model  uncertainty  is  closely  related  to  knowledge  uncertainty  in  that  it  poses  the 
question,  “are  we  thinking  about  this  system  correctly?”  In  other  words,  do  the  models  used  truly  represent 
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what  is  going  on  with  the  phenomena  of  interest?  Model  uncertainty  includes  the  limits  of  our  knowledge 
and  theory  as  well  as  the  specific  forms  of  the  models  we  use  to  represent  complex  relationships. 

Many  kinds  of  model  uncertainty  can  be  envisioned  in  a  HEP  analysis.  There  will  be  considerable 
uncertainty  about  the  choice  of  target  species.  There  is  some  question  about  whether  the  life  requisites  are 
truly  understood.  The  methods  by  which  suitability  indices  are  combined  to  form  a  habitat  suitability  index 
are  open  to  vigorous  debate.  The  major  appeal  of  the  process  may  be  more  its  acceptance  than  its  accuracy. 


It  is  not  difficult  to  envision  uncertainty 
about  the  values  of  the  quantities  that  have  to  be 
entered  into  the  HEP  model.  It  is  much  more 
difficult  to  try  to  think  about  the  uncertainty  in  the 
model  itself  How  much  uncertainty  is  there  in  the 
form  of  the  model?  How  well  developed  is  the 
theory  behind  the  HEP  model?  Are  the 
assumptions  underlying  the  model  reasonable? 
Unassailable?  How  much  uncertainty  is  there  in 
the  validity  of  identifying  an  Indicator  species  to 
represent  the  functioning  of  an  ecosystem?  How 
much  uncertainty  is  there  in  tying  the  future 
viability  of  a  species  to  a  habitat  suitability  index? 
Are  these  even  proper  ways  to  think  about  the 
problems? 

These  kinds  of  uncertainties  are  far  more 
difficult  to  think  about  and  confront  and  they  are 
often  overlooked  as  matter  of  convenience  if  not 
fear  of  what  a  critical  analysis  would  reveal  about 
our  favorite  models.  Uncertainty  about  model 
form  frequently  reflects  a  lack  of  theory  or  data,  or 
perhaps  a  disagreement  among  experts  about  the 
underlying  scientific  or  technical  mechanisms  at 
work  in  the  system  we  are  trying  to  model.  Given 
the  complexity  and  fledgling  state  of  ecosystem 
restoration  planning  there  would  appear  to  be 
potential  model  uncertainty  present  in  many 
environmental  investment  planning  activities. 

The  biological  and  habitat  models 
currently  in  use  are  major  sources  of  uncertainty  in 
the  evaluation  of  environmental  resources 
investment  plans.  Lack  of  theory  and  knowledge 
about  how  even  simple  ecosystems  function  is 


Model  Uncertainty  In  An  HSI  Model 

It  may  be  easier  to  understand  the  nature  of 
model  uncertainty  with  a  brief  example.  Consider 
the  U.S.  Department  of  the  Interior  Fish  and  Wildlife 
Service’s  HSI  model  for  the  mottled  duck.  The 
preface  describes  the  model  as  “. . .  a  hypothesis  of 
species-habitat  relationships,  not  a  statement  of 
proven  cause  and  effect  relationships.”  This  is  a 
clear  admission  of  model  uncertainty. 

The  model  uses  a  variety  of  literature  to 
identify  eight  specific  habitat  variables.  Might  there 
be  nine  or  more?  Are  these  the  eight  most  important 
variables?  These  are  all  legitimate  questions  that 
point  up  the  nature  of  model  uncertainty.  The 
suitability  graphs  linking  the  percentage  of  a  habitat 
variable  present  and  the  suitability  of  such  a 
percentage  expressed  as  a  zero  (low)  to  one  (high) 
index  are  linear  or  piecewise  linear  relationships. 

Many  scientists  would  agree  that  few  biological 
relationships  can  be  accurately  described  by  linear 
relationships.  This  is  another  potential  source  of 
model  uncertainty. 

Equations  used  to  combine  suitability 
indices  for  groupings  of  habitat  variables  into  cover 
values  involve  forms  like  cube  roots  of  three  factor 
products  and  other  presumably  arguable  weights  of 
the  relative  importance  of  the  various  habitat 
variables.  Likewise,  the  calculations  of  life  requisite 
values  and  ultimately  the  HSI  are  all  contestable 
formulations.  These  questions  raised  about  the 
manner  in  which  we  think  about  the  mottled  duck  are 
examples  of  potential  sources  of  model  uncertainty. 

--- .  .  ^ 
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another  common  source  of  model  uncertainty^  The  danger  of  model  uncertainty  often  lies  in  the  fact  that 
it  is  not  recognized.  Once  the  model  has  been  used  and  accepted  there  is  little  incentive  for  its  users  to  ask 
critical  questions  about  its  fundamental  “appropriateness”, 

SOURCES  OF  UNCERTAINTY  IN  EMPIRICAL  QUANTITIES 

Empirical  quantities  are  the  most  common  source  of  quantity  uncertainty  in  planning  studies.  What 
causes  the  uncertainty?  It’s  not  enough  to  identify  the  type  of  uncertainty;  we  need  to  know  something  about 
its  cause  so  that  the  uncertainty  can  be  addressed  in  an  appropriate  manner.  Some  common  sources  of 
uncertainty^  in  empirical  quantities  are  shown  in  Table  2.  Each  is  discussed  in  turn  below. 


Table  2:  Sources  of  Uncertainty  in  Empirical  Quantities 


Random  error  and  statistical  variation 


Systematic  error  and  subjective  judgment 
Linguistic  imprecision 


Variability 


Randomness  and  unpredictability 


Disagreement 

Approximation 


Random  Error  and  Statistical  Variation 

One  of  the  best  understood  sources  of  uncertainty  is  the  random  error  or  statistical  variation  that 
arises  from  errors  in  the  direct  measurement  of  a  quantity.  Few  measurements  are  exactly  right.  Our 
instruments  are  not  properly  calibrated,  our  powers  of  observation  are  imperfect,  we  are  flawed  human 
beings.  Our  tools  and  ourselves  make  mistakes.  These  are  random  errors.  Examples  of  this  type  of 
uncertainty  in  environmental  investment  will  include  estimates  of  the  extent  of  different  ground  covers, 
infiltration  of  water  through  soil  types,  or  fish  and  wildlife  counts. 

In  addition,  there  is  statistical  variation  that  results  from  random  processes  like  sampling  error. 
Suppose  the  average  depth  of  water  is  estimated  by  measuring  depths  at  randomly  selected  locations  in  a 
study  area.  Imagine  the  locations  were  selected  by  laying  a  grid  over  a  map  of  the  water  body  and  each  grid 
cell  was  uniquely  numbered  from  1  to  500.  If  40  cells  are  randomly  selected  and  the  average  depth  is 
computed  we  would  not  expect  the  same  average  depth  to  be  obtained  if  a  different  sample  was  taken.  In 


Most  of  the  discussion  of  quantity  and  model  uncertainty  as  well  as  the  discussion  that  follows  are  adapted  from  Uncertainty  A 
Guide  to  Dealing  with  Uncertainty  in  Quantitative  Risk  and  Policy  Analysis  by  M.  Granger  Morgan  and  Max  Henrion. 
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fact,  if  we  could  take  every  conceivable  combination  of  40  cells  from  the  population  of  500  we  would  have 
a  distribution  of  sample  means.  The  mean  of  this  distribution  of  sample  means  would  be  the  true  average 
water  depth. 

The  problem  is  that  when  we  take  one  sample  and  calculate  one  sample  mean  we  do  not  know  where 
in  that  theoretical  distribution  of  sample  means  our  estimate  comes  from.  The  “laws”  of  chance  suggest  our 
estimate  will  be  reasonably  close  to  the  true  mean  when  our  sample  is  large,  but  there  is  always  a  “dumb 
luck”  chance  we’ll  have  a  sample  mean  quite  different  from  the  true  mean.  Knowing  this  potential  for 
sample  error  exists,  it  must  be  dealt  with.  Inferential  statistics  provides  well  known  methods  for  doing  just 
that. 

Subjective  Judgments  and  Systematic  Errors 

Subjective  judgments  are  common  sources  of  systematic  errors.  So-called  experts  are  frequently 
ignorant  of  their  own  biases  in  estimating  or  assessing  quantitative  measures.  Experts  are  sometimes  the  last 
to  recognize  the  limits  of  their  experience  and  knowledge.  Even  when  experts  know  their  limitations,  their 
opinions  are  often  little  more  than  educated  guesses.  Guesses  are  uncertain  by  their  very  nature. 

Systematic  errors  arise  due  to  biases  in  the  measurement  instrument  or  the  individual  taking  the 
measurement.  Attempts  to  estimate  populations  from  animal  excrement  counts  can  lead  to  errors  if  the  ratio 
of  excrement  to  organisms  is  incorrect.  Estimating  values  with  inappropriate  or  poorly  calibrated  tools  yields 
systematic  errors.  Systematic  error  is  defined  as  the  difference  between  the  true  value  of  the  quantity  being 
measured  (the  actual  number  of  animals)  and  the  value  to  which  the  mean  of  the  measurements  converge 
as  more  observations  are  collected  (the  estimated  population  based  on  our  counting  method). 

Systematic  errors  can  occur  in  any  values  that  have  to  be  physically  measured  with  equipment  that 
needs  periodic  calibration  (e.g.,  hand-levels,  the  length  of  a  person's  pace,  flow  meters)  or  that  are  estimated 
by  people  unaware  of  their  imprecision  (e.g.,  habitat  suitability  indices).  The  solutions  to  these  types  of 
uncertainty  are  well-calibrated  equipment  in  sufficient  quantity  for  all  analysts  and  rigorous  training  in  data 
collection  techniques.  New  analysts,  contractors  with  minimally  trained  field  personnel,  and  inflexible 
experts  are  common  sources  of  such  errors. 

Linguistic  Imprecision 

Language  is  frequently  an  imprecise  vehicle  for  communication.  Filling  a  lake  shoreline  to  “minus 
one  foot”  with  dredge  material  sounds  very  precise  until  we  consider  that  one  agency  uses  mean  low  water, 
another  mean  sea  level,  and  a  third  national  geodetic  vertical  datum.  Though  everyone  is  hearing  the  same 
words,  they  are  all  thinking  something  quite  different.  Worse,  they  may  be  unaware  of  the  uncertainty 
because  they  understand  exactly  what  the  words  mean  to  them. 

In  many  cases,  a  bit  of  clear  thinking  and  precise  language  can  clear  up  most  uncertainty  stemming 
from  imprecise  communication.  Unfortunately,  when  a  new  jargon  is  developing,  as  it  is  with  environmental 
investments,  ecosystem  restoration  programs,  and  risk  analysis,  there  is  going  to  be  considerable  imprecision 
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in  the  language  and  considerable  uncertainty  about  its  meaning.  There  is  a  great  deal  of  variation  in  the  way 
different  people  interpret  words  and  phrases  and  their  interpretation  is  very  context-dependent. 

Even  words  in  common  usage  like  ecosystems,  resources,  and  functioning,  have  clouded  meanings 
in  new  contexts.  What  is  a  functioning  ecosystem?  Would  lumbermen  give  the  same  answers  an  Audubon 
Society  spokesperson  would  when  describing  the  functions  of  the  forests  of  the  Pacific  Northwest?  The 
linguistic  imprecision  that  permits  two  people  to  give  widely  varying  and  honest  definitions  to  the  same 
terminology  is  a  significant  source  of  uncertainty.  Fortunately,  linguistic  imprecision  can  be  relatively  easily 
avoided  once  its  potential  existence  is  recognized. 

Variability 

Many  quantities  vary  from  time-to-time  or  from  place-to-place.  The  flow  or  a  stream  through  a 
project  area  during  one  year  is  expected  to  be  different  from  the  flow  in  another  year.  The  birth  weight  of 
deer  in  one  area  may  differ  from  the  birth  weight  in  another  area.  Conceptually  we  might  have  a  high  degree 
of  certainty  about  a  frequency  or  probability  distribution.  However,  if  a  single  organism  is  randomly 
selected  from  that  population  it  may  differ  from  the  mean  because  of  the  variability  inherent  in  the 
population. 

Environmental  variables  of  interest  might  include  size  of  fish,  weight  of  deer,  flow  in  a  stream, 
salinity,  temperature,  tidal  fluctuations,  or  other  such  variables.  An  element  from  one  of  these  populations 
might  be  a  fish,  a  deer,  a  peak  annual  flow,  an  average  annual  flow,  a  salinity  or  temperature  reading,  and 
so  on.  Anyone  of  these  can  be  expected  to  differ  from  another  because  of  the  variability  that  exists  within 
the  population.  Uncertainty  due  to  variability  can  best  be  reduced  by  disaggregation. 

Randomness  and  Unpredictability 

Inherent  randomness  is  often  defined  as  uncertainty  that  is  impossible  to  reduce.  The  practical  issue 
for  the  analyst  is,  is  the  uncertainty  reducible  in  practice?  A  quantity  that  is  random  to  one  person  who  does 
not  understand  the  systems  at  work  may  be  deterministic  for  another  person  who  knows  the  underlying 
process  that  generated  the  quantity.  The  planner  should  try  to  distinguish  uncertainty  that  can  be  reduced 
by  further  study  from  that  which  is  inherently  random,  for  the  latter  cannot  be  reduced.  This  source  is  clearly 
related  to  the  type  of  uncertainty  due  to  lack  of  knowledge. 

Taking  full  advantage  of  the  expertise  present  in  the  interdisciplinary  study  team  can  help  reduce 
this  type  of  uncertainty  to  the  maximum  practical  extent.  Uncertainty  about  the  cost  of  constructing 
structures  can  often  be  reduced  by  more  detailed  estimating  techniques.  Uncertainty  about  the  years  in  which 
future  floods  will  occur  cannot  be  reduced  by  any  practical  means  and  it  is  not  worth  trying  to  do  so. 

Disagreement 

Different  people  can  view  the  same  problem  from  very  different  perspectives.  Two  experts  can  look 
at  the  same  evidence  and  draw  different  conclusions.  The  levees  have  prevented  millions  of  dollars  in 
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damages.  The  levees  have  destroyed  thousands  of  acres  of  wetlands.  Which  side  is  right?  When  experts 
disagree  about  a  quantity,  a  project’s  effect,  an  ecosystem’s  function,  or  any  other  critical  factor,  the  nature 
of  the  disagreement  should  be  carefully  explained  and  evaluated. 

Approximation 

Models  are  simplifications  of  complex  real  systems.  Tradeoffs  between  model  complexity  and 
computation  costs  dictate  that  most  model  results  yield  mere  approximations  of  the  actual  values.  The  only 
reliable  way  to  estimate  the  uncertainty  introduced  by  the  approximation  is  to  increase  the  resolution  of  the 
model  and  measure  the  difference  between  the  two  models.  In  other  words,  approximation  uncertainty  can 
be  difficult  to  ascertain.  It  is  most  often  addressed  by  sensitivity  analysis. 

Average  erosion  rates  based  on  comparison  of  aerial  photos  taken  25  years  apart  are  approximations. 
Depending  on  the  methods  used  many  quantities  can  be  approximations.  For  example,  analysts  who  base 
their  estimate  of  the  percent  of  unsubmerged  substrate  on  visual  inspection  are  using  approximations. 
Calculations  of  habitat  units  are  an  approximation.  Project  cost  estimates  are  approximated  early  in  the  study 
process.  How  good  are  the  approximations?  The  only  way  to  tell  is  to  compute  the  habitat  units  or  costs 
with  a  better  model.  This  is  rarely  possible.  If  a  better  model  were  available  it  would  have  been  used. 

Occasionally,  the  best  models  are  not  used  because  they  are  too  expensive.  A  reconnaissance  report 
may  quantify  project  outputs  in  acres  of  wetland  or  acres  of  a  restored  island.  The  feasibility  report  may  use 
a  HEP  analysis  to  better  quantify  the  project  outputs.  In  a  case  like  this,  the  uncertainty  due  to  approximation 
can  be  estimated  in  subsequent  studies  by  using  the  more  sophisticated  HEP  evaluation.  Likewise,  the 
uncertainty  in  cost  estimates  is  eliminated  as  projects  move  to  implementation. 

SUMMARY  AND  LOOK  FORWARD 

Environmental  planning  is  future-oriented  problem  solving  and  decision-making  intended  to  restore, 
preserve,  protect  and  enhance  environmental  resources  or  to  mitigate  unavoidable  damages  to  them.  As 
such,  it  is  fundamentally  decision-making  under  uncertainty.  Incorporating  risk  and  uncertainty  analysis  into 
environmental  planning,  wherever  it  is  done  in  the  organization,  will  improve  both  the  planning  process  and 
the  qualify  of  decisions  made  to  manage  the  nation’s  environmental  resources. 

Risk  and  uncertainty  analysis  is  easier  to  introduce  to  new  applications  when  there  is  a  structured 
and  logical  approach  to  the  process.  There  are  three  broad  types  of  uncertainty;  knowledge,  model  and 
quantity  uncertainty.  Quantity  uncertainty  is  the  most  commonly  encountered.  Understanding  the  nature 
of  the  uncertain  quantity  can  go  a  long  way  toward  prescribing  a  treatment  for  the  uncertainty.  Of  all 
quantities,  empirical  quantity  uncertainty  can  be  the  most  complex  to  deal  with,  because  there  are  at  least 
seven  different  causes  for  these  types  of  uncertainty.  The  taxonomy  offered  in  this  chapter  provides  the  new 
risk  analyst  with  a  way  to  think  about  the  uncertainty  that  is  present  in  environmental  planning  work. 

Although  the  taxonomy  offers  a  way  for  analysts  to  think  about  the  uncertainty  encountered  in 
environmental  planning  it  does  not  suggest  specific  methods  for  dealing  with  that  uncertainty.  That  is  done 
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generically  in  the  next  chapter  which  introduces  some  of  the  concepts  and  tools  used  by  risk  analysts. 
Subsequent  chapters  will  suggest  specific  applications  of  some  of  these  tools  to  environmental  planning 
situations,  both  generally  (Chapter  Five)  and  specifically  (Chapter  Six). 
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CHAPTER  FOUR:  POTENTIAL  RISK  AND 
UNCERTAINTY  TOOLS 

INTRODUCTION 

Once  important  sources  of  risk  and  uncertainty  are  identified  in  environmental  restoration  studies, 
what  can  be  done  about  them?  Fortunately,  there  are  a  wide  variety  of  tools  and  methods  available  for 
addressing  risk  and  uncertainty.  This  section  introduces  a  number  of  them.  The  chapter  is  divided  into  two 
major  sections,  following  this  introduction.  The  first  section  presents  some  broad  concepts  that  can  be  used 
to  introduce  risk  and  uncertainty  to  environmental  planning.  The  second  section  concentrates  on  more 
specific  tools. 

The  concepts  and  tools  presented  in  this  chapter  are  general  methods  that  can  be  applied  by  anyone, 
such  as  hydrologists,  economists,  environmental  scientists,  engineers,  and  others,  who  are  part  of  an 
interdisciplinary  team.  There  is  a  great  deal  of  range  and  diversity  in  the  uncertainty  analysts  will  encounter 
in  their  work.  There  are  also  a  large  number  of  tools  available  for  considering  much  of  this  uncertainty.  This 
chapter  presents  some  of  the  concepts  and  tools  in  common  usage  in  order  to  give  the  reader  a  feel  for  the 
nature  of  what  is  available  without  imparting  any  detailed  knowledge  about  their  usage.  This  means  at  times 
there  will  be  too  much  detail  for  some  readers  and  not  enough  for  others.  Additional  details  on  these 
techniques  are  readily  available  in  the  literature  or  form  standard  college-level  courses.  It  is,  perhaps,  most 
appropriate  to  think  of  this  chapter  as  a  partial  catalogue  of  tools  that  are  available  for  future  use  and 
adaptation  by  the  Corps  of  Engineers  as  it  integrates  the  use  of  risk  and  uncertainty  analysis  into  the 
environmental  planning  decision  process. 

BROAD  CONCEPTS 

Some  of  the  most  obvious  places  to  start  with  risk  and  uncertainty  analysis  are,  in  retrospect,  rather 
broad  and  obvious.  The  first  step  might  be  to  acknowledge  that  uncertainty  exists  and  then  to  begin  to 
educate  people  about  the  need  to  do  something  about  it.  This  education  process  will  require  communication, 
but  communication  about  risky  situations  has  some  unique  elements  to  it.  Acknowledgment,  education  and 
risk  communication  are  examples  of  some  of  the  broad  concepts  described  in  this  section  of  the  report. 
These  broad  concepts  are,  perhaps,  the  types  of  things  common  sense  would  lead  us  to,  given  the  luxury  of 
time  to  think  about  the  application  of  risk  and  uncertainty  analysis  to  environmental  planning.  A  number 
of  these  broad  concepts  are  presented  here  to  give  the  reader  the  benefit  of  the  common  sense  thinking  others 
involved  in  risk  analysis  have  already  done.  The  concepts  are  presented  in  alphabetical  order.  The  order 
of  significance  will  depend  on  the  circumstances  of  each  planning  activity. 

ACKNOWLEDGING  UNCERTAINTY 

The  simplest  place  to  start  in  incorporating  risk  and  uncertainty  analysis  into  environmental  planning 
is  to  explicitly  acknowledge  the  existence  of  uncertainty  in  the  analysis.  Far  too  often,  precision  is 
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substituted  for  accuracy.  Analysts,  failing  to  recognize  the  assumptions  and  limitations  of  their  data  and 
models,  often  present  results  as  if  they  are  far  more  certain  that  they  in  fact  are. 

It  may  be  far  more  honest  to  say  the  percent  of  unsubmerged  substrate  is  estimated,  with  95% 
confidence,  to  be  between  26%  and  34%  than  to  say  it  is  30%.  The  latter  value,  though  precise,  is  likely  to 
imply  an  accuracy/certainty,  that  does  not  likely  exist  in  reality.  It  is  not  a  weakness  to  be  uncertain,  it  is 
essential  honesty  in  many  cases.  Acknowledging  our  uncertainty  is  the  cornerstone  of  any  risk  and 
uncertainty  analysis. 

EDUCATION  AND  TRAINING 

The  first  and  most  important  tool  for  the  risk  and  uncertainty  toolbox  is  knowledge.  Risk  and 
uncertainty-based  analysis  uses  selected  methods  and  tools  that  may  or  may  not  be  known  to  Corps  analysts. 
You  can’t  do  water  resources  planning  without  knowledge  of  the  hydrologic  cycle  and  you  can’t  do  risk- 
based  analysis  without  some  knowledge  of  probability,  statistics,  and  the  emerging  literature  on  risk  and 
uncertainty. 

People  are  not  born  knowing  these  things  and  if  Corps  analysts  are  going  to  be  expected  to  apply 
these  techniques  they  must  first  be  taught  them.  The  necessary  knowledge  can  be  obtained  in  a  variety  of 
ways.  There  is  a  vast  and  growing  literature  treating  probability,  statistics  and  risk  and  uncertainty  analysis. 
Incorporating  Risk  and  Uncertainty  into  Environmental  Evaluation:  An  Annotated  Bibliography  (IWR 
Report  96-R-9  March  1996)  provides  an  excellent  introduction  to  some  of  this  literature.  These  are  topics 
that  many  people  find  difficult  to  pick  up  on  their  own,  however. 

The  standard  college-level  introduction  to  statistics  courses  will  provide  the  rudimentary  knowledge 
of  probability  and  statistics  that  is  required.  These  courses  frequently  emphasize  descriptive  statistics, 
interval  estimation,  and  hypothesis  testing  while  glossing  over  or  ignoring  distribution  fitting  and  other 
topics  more  useful  to  Corps  risk  analysts.  Application-specific  training  such  as  the  Corps  currently  provides 
in  a  number  of  areas,  including  a  risk  analysis  course,  may  be  the  most  effective  means  of  providing  Corps 
personnel  with  the  requisite  skills  for  conducting  risk-based  analyses.  Once  trained.  Corps  field  personnel 
will  be  perfectly  capable  of  applying  the  tools  and  techniques,  extending  their  usage  and  finding  new  and 
innovative  applications  in  the  field  of  environmental  planning.  It’s  not  reasonable  to  expect  that  each  Corps 
analyst  will  develop  knowledge  and  expertise  in  every  aspect  of  risk  and  uncertainty  analysis.  The  necessary 
expertise  can  frequently  be  acquired  through  other  people.  This  can  mean  consultation  with  other  Corps 
experts,  interpersonnel  agreements  with  agencies/organizations  outside  the  Corps,  or  contracting  for  the 
services  of  a  consultant.  Regardless  of  the  preferred  method  for  acquiring  the  necessary  knowledge, 
acquiring  it  is  an  essential  step  in  dealing  with  risk  and  uncertainty  analysis. 

ERA’S  ECOLOGICAL  RISK  ANALYSIS  PARADIGM 

At  this  writing  there  is  not  yet  a  final  ecological  risk  paradigm.  It  is  under  development. 
Nonetheless,  the  development  of  this  paradigm  bears  watching.  The  existing  environmental  risk  assessment 
paradigm  is  already  finding  useful  applications  where  the  concern  is  the  effect  of  toxic  substances  on  one 
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or  more  species.  As  noted  earlier,  the  Corps  dredged  material  disposal  agreements  with  EPA  make  some 
use  of  that  paradigm. 

The  appeal  of  the  ecological  risk  assessment  paradigm  is  that  it  will  employ  a  stress-response 
concept.  That  is,  substances  or  events  that  introduce  stress  to  an  ecosystem  will  be  identified  and  the 
ecosystem’s  response  to  those  stressors  will  be  quantified.  The  reliance  on  stressors  will  make  this  paradigm 
much  more  amenable  to  the  evaluation  of  Corps  projects  because  changes  in  stream  flow  velocity,  disruption 
of  sediment  budgets,  and  the  many  perturbations  of  the  natural  hydrologic  cycle  that  can  result  from  Corps 
projects  do  stress  ecosystems.  The  ecosystems  orientation  is  also  more  appealing  than  a  single  species 
approach. 

It  is  suggested  that  Corps  analysts  keep  abreast  of  what  EPA  is  doing  in  this  area.  Both  the  Corps 
and  EPA  are  struggling  with  many  of  the  same  issues  of  target  species  selection,  extrapolation  of  analytical 
results,  habitat  loss  and  higher  level  systems  like  ecosystems  and  the  global  environment. 

FULL  DOCUMENTATION 

Full  documentation  of  the  risk  and  uncertainty  analyses  is  very  important,  particularly  during  the 
early  stages  when  these  techniques  are  first  being  applied.  Describing  the  initial  uncertainty,  the  steps  taken 
to  reduce  it,  and  the  uncertainty  that  remained  are  essential  steps  in  a  full  documentation.  Describing  the 
hazards,  probabilities  and  consequences  of  risky  situations  are  equally  important. 

A  complete  documentation  is  important  to  the  related  tasks  of  risk  communication  and  peer  review. 
There  are  other  reasons  for  documenting  the  analysis.  Positive  examples  of  how  to  document  analysis  are 
always  welcome.  Although  many  analyses  are  case  specific  there  may  be  components  of  the  analysis  or 
results  that  can  be  used  elsewhere. 

A  well-documented  analysis  aids  the  decision  process.  The  report  should  not  banish  documentation 
of  the  risk  and  uncertainty  analysis  to  a  brief  section  at  the  end  of  the  report.  If  the  problems  are  uncertain 
or  risky,  the  analysis  should  be  presented  when  the  problems  are  described.  Descriptions  of  sampling  results, 
subjectively  estimated  data  and  model  elements,  rationales  for  assumptions  and  leaps  of  faith,  justifications 
of  decisions  and  the  like  should  be  documented  throughout  the  study  report.  In  some  cases  it  may  be 
appropriate  to  summarize  the  results  of  more  formal  risk  and  uncertainty  analyses,  for  example,  simulations 
or  sensitivity  analyses  that  could  affect  the  decision,  in  a  separate  section  of  the  report.  Careful 
documentation  can  be  an  effective  tool  for  the  internal  and  external  two-way  flows  of  information  and 
feedback  that  are  so  essential  to  a  successful  project. 

JARGON  AND  LITERATURE 

Risk  has  been  described  as  a  subset  of  uncertainty.  Risk  analysis,  however,  is  a  term  in  general 
usage  that  includes  the  analysis  of  uncertain  situations.  Hence,  we  find  the  cumbersome  situation  in  which 
the  jargon  is  quite  confusing.  Risk  is  a  subset  of  uncertainty,  but  risk  analysis  includes  uncertainty  analysis? 
The  way  the  language  is  used,  that  is  correct!  Be  forewarned,  the  language  of  risk  and  uncertainty  can  be 
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uncertain.  When  the  distinctions  are  important  it  is  best  to  take  the  time  needed  to  clarify  meanings 
precisely.  Fortunately,  there  will  not  likely  be  many  situations  in  which  the  distinctions  are  that  important. 

There  is  a  dichotomous  nature  to  risk  analysis  that  is  important  and  that  bears  repeating.  There  are 
the  technical/scientific  aspects  of  risk  and  uncertainty  and  there  are  the  policy  issues  of  risk  and  uncertainty. 
Often  these  two  worlds  are  not  well-suited  to  mutual  communication.  The  rational  world  of  science  is  ill- 
prepared  to  address  the  irrational  fears  and  emotions  of  humans.  Decision-makers  are  often  unable  to  absorb 
the  real  meaning  of  a  technical  analysis.  Risk  analysis  is  a  process  that  includes  the  assessment  and 
management  of  risk  information  for  an  individual,  group,  society  or  environment.  It  is  the  combination  and, 
hopefully,  the  synthesis  of  these  two  tasks. 

One  part  of  a  risk  analysis  is  the  technical  or  scientific  part.  This  is  presumably  an  objecftve 
assessment  of  the  hazards,  their  probabilities  and  consequences  handled  by  qualified  experts.  This  is  a 
detailed  investigation  performed  to  understand  the  nature  of  unwanted,  negative  consequences.  If  s  an 
analytical  process  that  provides  information  regarding  undesirable  events  by  quantifying,  to  the  extent 
possible,  their  probabilities  and  their  consequences. 

Once  risks  have  been  objectively  assessed  the  limits  of  science  have  been  reached  and  now  social 
and  political  values  are  required  to  decide  what  to  do  about  the  assessed  risks.  Risk  management  is  the 
decision-  making  or  management  process  used  to  protect  human  life,  health,  property  or  the  environment 
from  unwanted  adverse  consequences.  The  management  step  of  the  risk  analysis  expressly  weighs  ethical, 
social  and  political  values. 

The  risk  assessment  may  identify  a  10%  to  30%  chance  that  beaver  will  become  locally  extinct  in 
the  next  20  years.  The  risk  management  must  determine  whether  or  not  that  is  an  acceptable  risk.  If  it  is  not, 
it  must  decide  what,  if  anything,  to  do  about  it. 

There  is  not  a  well-defined  discipline  called  risk  and  uncertainty  analysis.  Risk  analysis  comprises 
a  set  of  tools  and  methods  developed  in  and  borrowed  from  a  variety  of  disciplines.  Nonetheless,  this 
collection  of  tools  and  methods  have  enough  common  elements  that  risk  analysis  has  evolved  into  a  specialty 
subfield  of  many  disciplines.  Risk  analysis  has  its  own  language,  its  own  theory,  and  its  own  paradigms. 

Corps  personnel  doing  risk  analysis  need  to  understand  some  of  the  basic  language  of  this  subfield, 
because  it  is  not  always  precise.  Different  authors  and  agencies  use  the  same  terms  to  convey  different 
meanings.  Analysts  need  to  be  familiar  with  terms  like  expected  utility,  acceptable  risk,  risk  assessment, 
environmental  risk,  ecological  risk,  the  EPA  paradigm,  expected  values,  partitioned  risk,  comparative  risk 
analysis,  risk-cost  and  benefit-risk  trade-offs,  risk  communication  and  the  like. 

There  is  a  great  deal  of  existing  and  emerging  literature  on  risk  analysis.  This  report  refers  the  reader 
to  the  following  books  as  a  good  place  to  start. 

1)  Managerial  Decisions  Under  Uncertainty:  An  Introduction  to  the  Analysis  of 
Decision-Making.  By  Bruce  F.  Baird.  New  York:  John  Wiley  &  Sons,  1989. 
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2)  Making  Hard  Decisions:  An  Introduction  to  Decision  Analysis.  By  Robert  T. 

Clemen.  Boston,  MA:  PWS-Kent  Publishing  Company,  1990. 

3)  Readings  in  Risk.  Edited  by  Theodore  S.  Glockman  and  Michael  Gough. 
Washington,  D.C.:  Resources  for  the  Future,  1993. 

4)  Uncertainty:  A  Guide  to  Dealing  with  Uncertainty  in  Quantitative  Risk  and  Policy 
Analysis.  By  M.  Granger  Morgan  and  Max  Henrion  with  a  chapter  by  Mitchell 
Small.  New  York  and  Cambridge:  Cambridge  University  Press,  1992. 

5)  Simulation  and  Modeling  and  Analysis.  By  Averil  M.  Law  and  W.  David  Kelton. 

New  York,  NY:  McGraw-Hill,  Inc.,  1991. 

6)  “Reducing  Risk:  Setting  Priorities  and  Strategies  for  Environmental  Protection.”  By 
Science  Advisory  Board  (A- 101),  SAB-EC-90-021.  The  Report  of  the  Science 
Advisory  Board:  Relative  Risk  Reduction  Strategies  Committee  to  the  U.S. 
Environmental  Protection  Agency.  Washington,  D.C.:  USEPA,  September  25, 

1990. 

Basic  awareness  of  the  jargon  and  developments  in  the  risk  and  uncertainty  literature  is  essential. 

PEER  REVIEW 

An  independent  review  of  any  risk  analysis  can  be  an  effective  check  on  the  integrity  and  credibility 
of  a  risk  analysis.  Peer  review  of  risk  analyses  is  a  central  part  of  many  of  the  risk  analysis  legislation 
initiatives  now  before  Congress.  Legislative  requirements  aside,  whenever  risk  analysis  is  an  important  part 
of  a  project  with  potential  outrage  factors  (see  section  on  Risk  Communication  that  follows)  that  could  affect 
the  viability  and/or  performance  of  a  plan,  peer  review  can  be  an  effective  tool.  This  will  be  particularly  so 
for  studies  with  international  or  significant  intercultural  effects. 

Consider  a  recent  example.  It  had  been  proposed  that  the  channel  catfish  be  introduced  to  New 
Zealand  via  fish  farming,  not  in  the  wild.  An  environmental  impact  assessment  considered  the  consequences 
of  these  fish  escaping  to  the  wild  and  concluded  the  risks  would  be  tolerable.  Public  concern  resulted  in  a 
peer  review  that  disclosed  documented  cases  of  catfish  escapes  from  farms  to  the  wild.  Its  rapid  spread  to 
various  habitats  and  the  resulting  damage  through  predation  and  competition  with  indigenous  species  were 
enough  to  halt  the  introduction  of  the  species. 

More  points  of  view  and  fuller  information  are  obvious  benefits  of  a  peer  review.  The  environmental 
impact  statement  (EIS)  review  by  the  public  has  been  designed  to  assure  these  benefits.  Corps  studies  have 
always  been  subjected  to  a  peer  review  of  sorts  in  recent  decades.  That  review  has,  traditionally,  been  an 
in-house  review  by  other  agency  personnel.  As  the  Corps’  review  process  continues  to  evolve,  independent 
peer  review  as  appropriate  may  prove  a  valuable  feature  of  controversial  risk  and  uncertainty  analyses, 
especially  in  environmental  restoration  projects. 
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POST  FACTO  EVALUATION 

Environmental  restoration  projects  are  relatively  new  in  concept.  There  is  much  that  is  not  known 
about  the  environments’  we  seek  to  restore  and  the  methods  by  which  this  might  be  best  accomplished. 
Monitoring  would  ideally  be  part  of  every  project  design.  There  is  a  tremendous  need  for  information  about 
the  results  of  these  projects.  Were  objectives  achieved?  Can  errors  in  project  design  be  corrected  to  avoid 
catastrophe? 

One  of  the  most  effective  ways  to  reduce  uncertainty  is  through  information.  Monitoring  and 
observing  the  performance  of  projects  after  they  are  built  and  disseminating  that  information  broadly  is  one 
of  the  most  efficient  uses  of  project  funds  that  could  conceivably  be  made.  Monitoring  might  include  visual 
inspection,  mapping,  bathymetry,  aerial  photographs,  water  quality  monitoring,  fish  and  other  species 
monitoring,  field  inspection,  GIS  mapping,  and  other  methods. 

Lessons  learned  with  environmental  restoration  projects  will  be  worth  their  weight  in  gold. 
Newsletters,  conferences,  seminars,  training  programs,  world  wide  web  site  home  pages,  mailing  lists, 
bulletin  boards,  news  groups,  Internet  relay  chat  (IRC),  teleconferences,  video  cassettes,  multi-media 
presentations,  photo  libraries,  and  repository  libraries  for  restoration  projects  are  but  some  of  the  means  by 
which  valuable  information  can  be  disseminated  and  uncertainty  reduced. 

PUBLICITY 

Publicity  can  be  an  important  tool  in  risk  and  uncertainty  analysis.  Take  something  as  basic  as  the 
recent  conditions  at  a  project  site  as  an  example.  A  typical  analysis  most  often  involves  either  a  literature 
search  or  a  comparison  of  old  aerial  photographs  to  the  current  condition  to  determine  what  changes  have 
been  taking  place.  Publicity  can  be  very  helpful  in  uncovering  the  maximum  possible  information.  There 
may  be  obscure  local  journals  or  publications  with  valuable  information  of  which  Corps  personnel  are 
unaware.  Personal  histories  from  long-time  residents  may  be  important  sources  of  information.  The  mere 
act  of  publicizing  the  need  for  information  about  a  place  where  many  interested  people  live  can  provide 
valuable  information. 

In  the  case  of  risky  situations,  publicity  can  provide  an  opportunity  for  all  sides  to  come  out  and  be 
heard.  Publicity  can  be  provided  through  formal  means  like  meetings,  newsletters  and  the  like  or  through 
informal  means  like  talk  radio  appearances,  conversations  with  historical  societies,  community  groups  and 
the  like.  Personal  communications  can  be  a  very  effective  part  of  a  publicity  campaign  to  reduce  uncertainty 
and  deal  with  risk. 

RISK  COMMUNICATION 

Communication  is  always  a  critical  element  in  any  planning  activity.  In  its  absence,  lack  of 
information  and  uncertainty  is  the  default  condition.  Risk  communication,  however,  refers  to  something 
more  specific.  It  is  communication  about  situations  in  which  risk  is  important.  That  communication  may 
be  among  team  members,  between  Federal  and  non-Federal  partners,  between  risk  assessors  and  risk 
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evaluators,  communication  with  the  media,  or  communication  with  the  public.  The  planner  needs  to  consider 
the  four  elements  of  the  risk  communication  process: 

•  Objective(s):  Why  is  the  communication  being  undertaken? 

•  Content  of  the  message:  What  information  is  to  be  conveyed  in  order  to  accomplish 
the  objectives? 

•  Form  of  communication;  How  should  the  message  be  transmitted  from  the  source 
to  the  receiver? 

•  Feedback  from  the  audience;  What  is  being  received? 


There  has  been  much  debate  about  what  defines  an  acceptable  risk.  In  fact,  it  may  be  that  risk  is  not 
accepted  or  acceptable  in  the  sense  that  it’s  consciously  balanced  against  a  perceived  good  as  much  as  it  is 
tolerated  where  necessary.  Generally,  people  are  risk  adverse.  Risks  are  perceived  as  intolerable  when 
specific  “outrage  factors”  are  present.  Risks  tend  to  be  less  tolerable  if  they  include  the  characteristics  shown 
in  Figure  4. 


Environmental  projects  can  be 
controversial  and  subject  to  strong 
emotional  reactions.  Environmental  risks 
are  subject  to  outrage  factors.  This  makes 
risk  communication  all  the  more 
important.  The  Corps  of  Engineers  is 
confronted  with  past  projects,  now  taken 
out  of  their  historical  context  by  some 
interest  groups  and  stakeholders.  Even 
when  a  restoration  project  succeeds  there 
will  be  some  who  feel  the  Corps  has 
failed  by  not  doing  more. 

It  is  important  that  the 
perceptions  of  these  and  other  groups, 
particularly  as  they  concern  the  risks  of  a 
project,  be  considered  part  of  the  planning  process  and  not  a  problem  to  be  overcome.  For  example,  if  some 
people  are  concerned  that  a  species  may  become  locally  extinct,  or  that  a  project  will  fail  to  result  in  the 
habitat  quality  expected,  these  are  not  things  that  people  should  be  talked  out  of,  i.e.  problems  to  overcome. 
They  are  legitimate  concerns  of  the  public  that  need  to  be  adequately  addressed  in  the  investigation.  The 
results  of  the  analysis  must,  likewise,  be  effectively  communicated.  As  a  general  rule,  when  discussing 
uncertain  situations  it’s  wise  not  to  use  untested  communications.  Take  care  when  discussing  risk  in  the 
public  sphere. 
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SOFTWARE 

Risk  analysis  software  comes  from  four  different  sources.  First,  there  is  the  commercial  software 
developed  for  sale  in  the  retail  market.  These  include  such  things  as  @RISK  and  Crystal  Ball.  Second,  there 
are  special  application  proprietary  tools  and  models  developed  privately  or  commercially  for  very  specific 
applications.  These  include  efforts  like  HEC’s  new  risk-based  expected  annual  damage  estimating  software. 
Third,  there  is  shareware  that  is  available  free  on  the  Internet  and  through  other  on-line  sources.  Fourth, 
there  is  the  do-it-yourself  software  that  are  individual  models,  developed  for  specific  studies,  by  individuals, 
that  are  not  intended  for  multiple  uses. 

The  first  class  of  software  tools  has  become  quite  large.  They  include  Monte  Carlo  and  Latin 
Hypercube  simulation  modeling  techniques  (e.g.,  @RISK,  Crystal  Ball,  FuzziCalc,  Predict!),  analytical 
hierarchy  process  multi-criteria  decision  models  (e.g.,  Expert  Choice),  decision  tree  and  influence  diagram 
packages  (e.g.,  DPL,  DATA),  visual  spreadsheets  (e.g.,  DS  Lab),  neural  networks  (Neuralyst,  Braincel), 
systems  dynamics  (e.g.,  Powersim,  ithink.  Optima,  Logical  Decisions,  Stella),  distribution  fitting  (e.g., 
BestFit),  linear  and  nonlinear  programming  (e.g.,  What’s  Best!,  Evolver,  Lindo)  and  traditional  statistics 
packages  (e.g.,  SPSS  for  Windows,  Statgraphics,  eHOP).  Two  sources  of  information  are  industry 
catalogues  produced  for  and  by  the  makers  of  this  software:  “Analytical  PowerTools’V  and  “SciTech 
Software  for  Science”.^ 

The  second  type  of  software  has  been  developed  for  specific  applications  but  has  found  a  limited 
niche  market.  At  times  this  software  has  been  developed  for  a  government  agency  and  its  availability  is 
subject  to  agency  regulation.  In  other  cases  the  developer  maintained  control  of  the  software  and  it  is 
available  for  sale.  This  occasionally  includes  adaptation  for  the  buyer.  This  type  of  software  is  sometimes 
substantially  more  expensive  than  software  available  on  the  retail  market. 

Examples  of  this  software  include  Systems  Analysis  Programs  for  Hands-On  Integrated  Reliability 
Evaluations  (SAPHIRE5.12),  a  fault  tree  package  originally  developed  for  the  U.S.  Nuclear  Regulatory 
Commission.  Much  of  this  software  has  been  developed  for  the  health  and  safety  aspects  of  risk  analysis. 

There  is  shareware  and  free  software  available  from  the  Internet.  These  are  of  varying  quality.  One 
example  is  the  risk/exposure  assessment  model  developed  for  the  EPA  by  the  University  of  Nebraska 
available  from  the  World  Wide  Web  (at:  http://eeyore.lv-hrc.nevada.edu/therdbase.html)  at  no  cost.  There 
are  numerous  mailing  lists  and  home  page  web  sites  detailing  these  opportunities.  A  great  number  of  these 
are  software  for  processing  graphical  belief  functions,  e.g.,  BAYES,  BELIEF  1.2,  IDEAL,  +  MacEvidence, 
+  Pulcinella,  +  SPI,  +  tresBel,  and  XBaies. 


Available  from  Palisade  Corporation  at  1-800-432-7475. 


Available  from  SciTech  International,  Inc.  at  312-486-9191. 
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The  final  category  of  risk  software  includes  models  developed  for  specific  project  evaluations. 
These  are  normally  developed  by  individuals  working  for  the  Corps  in  order  to  meet  the  specific  needs  of 
a  project.  Currently,  there  is  no  systematic  cataloging  of  these  models.  Creating  a  Corps  news  group  or 
mailing  list  where  such  information  can  be  exchanged  would  be  a  valuable  early  step  in  facilitating  the 
spread  of  risk  analysis  throughout  the  Corps’  environmental  planning  program. 

One  particularly  useful  source  of  information  about  software  and  other  risk-related  topics  is  a  risk 
analysis  mailing  list  maintained  on  the  Internet  by  Batelle  Corporation.  The  e-mail  address  for  this  mailing 
list  is  riskanal@bbs.pnl.gov.  This  mailing  list  can  put  any  risk  analyst  in  instant  contact  with  hundreds  or 
thousands  of  other  risk  analysts  around  the  world, 

SPECIFIC  TOOLS 

In  addition  to  the  broad  concepts  of  the  preceding  section  there  are  a  great  variety  of  specific  tools 
used  by  risk  analysts.  A  basic  familiarity  with  the  existence  of  these  methods  and  tools  can  be  an  invaluable 
aid  to  an  analyst  grappling  with  the  introduction  of  risk  and  uncertainty  analysis  to  a  new  domain  area.  It 
is  not  necessary  to  be  an  expert  in  all  or  even  any  of  these  techniques,  as  that  expertise  can  be  acquired  by 
training  or  contract.  It  is,  however,  important  to  have  some  awareness  of  the  range  of  tools  used  by  risk 
analysts.  What  follows  is  another  alphabetized  list  of  selected  tools  in  common  usage  among  risk  analysts. 
Several  of  them  will  be  used  in  the  example  in  Chapter  Six. 

ANALYTICAL  SOLUTIONS 

Linear  programming,  integer  programming,  nonlinear  programming,  dynamic  programming, 
calculus,  algebra,  and  game  theory  are  some  tools  frequently  used  to  solve  problems.  These  tools  yield 
analytical  solutions  to  problems.  Analytical  solutions  are  solutions  that  meet  all  the  criteria  of  a  problem 
specification.  The  analytical  solution  to  2  -t-  x  =  4  is  x  =  2.  Some  problems  have  more  than  one  analytical 
solution,  for  example,  2x  +  y  =  10  has  both  (2,6)  and  (0,10)  among  its  solution  set.  In  more  complex 
problems  we  are  often  interested  in  the  solution  that  optimizes  a  value.  We  may  be  interested  in  the 
analytical  solution  that  makes  net  benefits  as  large  as  possible  or  stream  velocity  as  small  as  possible. 

Some  complex  systems  can  be  formulated  as  models  that  can  be  solved  analytically.  The  goal  of 
most  of  those  models  is  to  determine  an  optimal  solution  through  the  use  of  algorithms.  For  example,  linear 
programming  problems  can  be  solved  analytically.  Not  all  problems  can  be  represented  adequately  by  such 
models.  Some  systems  are  too  complex  and  they  defy  analytical  solutions. 

Where  analytical  models  represent  reality,  simulation  simply  imitates  it.  Simulation,  discussed 
below,  imitates  the  operation  of  a  real  world  system  over  time.  It  is  a  numerical  technique  that  can  be  used 
to  estimate  the  analytical  solution  to  a  problem.  Simulation  is  not  an  optimization  technique.  The  answers 
obtained  from  simulations  are  not  analytical  solutions.  Solve  the  equation  2  +  x  =  4  a  thousand  times  and 
the  answer  will  always  be  the  same,  x  =  2.  Simulations,  on  the  other  hand  could  yield  many  different 
answers  to  a  question. 
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Analytical  solutions  are  preferred  whenever  they  can  be  obtained  so  long  as  the  models  from  which 
they  are  derived  adequately  represent  the  real  world  system  being  analyzed.  Risk  analysis  is  a  new  concept 
to  many  Corps  analysts.  The  first  exposure  for  many  people  is  a  Monte  Carlo  process  such  as  is  used  for  the 
estimation  of  expected  annual  damages  for  flood  damage  reduction  studies.  A  limited  exposure  to  risk 
analysis  tools  and  methods  leads  some  to  believe  that  risk  analysis  is  synonymous  with  simulation.  It  is  not. 

Even  when  risk  and  uncertainty  is  involved,  there  is  often  an  analytical  solution  to  a  problem. 
Suppose  you  are  a  gambler  and  you  face  a  game  based  on  the  toss  of  a  fair  coin.  If  it  comes  up  heads  you 
lose  $20  and  if  it  comes  up  tails  you  win  $10.  Is  this  a  game  you  want  to  play? 

We  don’t  need  a  simulation  or  any  other  fancy  tools  and  methods  to  answer  this  question.  The 
expected  value  of  this  game’s  result  can  be  calculated.  It  is  0.5(-$20)  +  0.5($10)  =  -$5.  On  average  you 
expect  to  lose  $5  per  game  if  you  play.  Clearly  it  is  not  worth  playing. 

Many  risky  or  uncertain  situations  are  far  more  complex  than  this  simple  example.  The  important 
point,  however,  is  that  some  problems  have  solutions  that  can  be  determined  through  analytical  solutions. 
“Analytical  Solutions”  is  included  here  to  assure  that  Corps  personnel  do  not  make  the  mistake  of  thinking 
that  simulation  and  Monte  Carlo  processes  are  always  the  best  approach  to  a  problem.  An  analytical  solution 
is  always  to  be  preferred  when  one  is  available.  Unfortunately,  they  are  not  often  available  or  they  are  too 
complex  to  solve.  Another  limitation  of  analytical  solutions  is  that  they  often  require  simplifying 
assumptions  in  order  to  make  them  solvable.  These  simplifying  assumptions  can  often  limit  the  usefulness 
of  the  resulting  analytical  solution. 

DECISION  CRITERIA 

Under  uncertainty  there  might  be  several  different  estimates  of  a  value  of  interest.  For  example,  a 
sensitivity  analysis  could  lead  to  three  or  more  estimates  of  the  number  of  habitat  units  (HU’s)  that  would 
result  from  a  project.  If  we  elicit  subjective  probability  estimates  (see  section  on  Expert  Opinion)  fi-om  three 
experts  we  can  expect  three  different  answers.  What  do  you  do  when  you  have  multiple  estimates  of  an 
important  value?  How  do  you  make  a  decision? 

There  are  a  number  of  well  known  decision  criteria  that  can  be  used  in  decision-making  under 
uncertainty.  At  this  point  the  Corps  has  not  taken  a  position  on  any  of  these  criteria,  so  presumably  the 
criterion  that  best  suits  the  needs  of  the  planning  partners  can  be  used.  To  facilitate  the  discussion  assume 
three  experts  have  identified  the  most  likely,  the  minimum,  and  maximum  number  of  white  tail  deer  that 
would  remain  in  the  study  area  50  years  from  now  if  no  action  is  taken.  These  hypothetical  estimates  are 
presented  in  Table  3  and  are  used  to  illustrate  the  use  of  the  selected  criteria  that  follow.'® 


For  more  information  about  these  criteria  see  “Guidelines  for  Risk  and  Uncertainty  Analysis  in  Water  Resources  Planning 
Volume  1  Principles  with  Technical  Appendices,  Appendix  I”,  March  1992,  IWR  Report  92-R-l. 
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Table  3:  Hypothetical  Estimates  of  White  Tail  Deer  Population 

Expert  1 

Expert  2 

Expert  3 

Minimum 

0 

50 

30 

Most  Likely 

21 

75 

300 

Maximum 

45 

100 

350 

Laplace  Criterion 

The  probability  of  Expert  I’s  outcome  being  realized  is  unknown,  as  is  the  probability  of  any  other 
expert's  outcome  being  realized.  When  the  probabilities  of  the  possible  outcomes  (i.e.,  the  ‘'partial 
probabilities”)  are  not  known,  equal  probabilities  can  be  assigned  to  all  the  outcomes.  The  Laplace  criterion 
assigns  each  of  the  outcomes  an  equal  probability.  With  three  estimates  of  the  most  likely  population  each 
outcome  receives  a  weight  of  Vs.  The  result  of  the  Laplace  criterion  is  an  estimated  population  of  V3(21)  + 
V3(75)  +  y3(300)  or  132  deer.  This  is  a  generally  accepted  default  criterion. 

Maximin  Criterion 

The  maximin  criterion  yields  the  “best”  of  the  worst  potential  outcomes.  The  worst  outcomes  are 
the  minimum  population  estimates.  The  best  of  the  worst  outcomes  is  the  one  that  has  the  largest  minimum 
population  estimate,  in  this  case  it  is  Expert  2's  estimate  of  50  deer.  This  emphasizes  the  worst  case  set  of 
outcomes  but  takes  an  optimistic  view  of  these  outcomes. 

Minimin  Criterion 

Under  this  criterion  we  choose  the  smallest  of  the  minimum  values.  It  is  the  most  pessimistic 
estimate  of  all  the  population  estimates  offered  here.  The  so-called  “minimin”  criterion  is  zero  deer.  This 
is  the  most  severe  criterion  and  may  be  favored  when  the  worst  case  outcome  is  unacceptable. 

Maximax  Criterion 

This  is  the  opposite  of  the  minimin  criterion.  It  considers  only  the  maximum  population  estimates 
and  it  uses  the  highest  of  these  estimates.  The  maximax  criteria  yields  a  population  estimate  of  350  deer. 
This  criterion  is  often  preferred  by  those  who  want  to  stress  the  most  optimistic  outcome. 

Minimax  Criterion 

The  minimax  criterion  selects  the  estimate  that  is  the  lowest  of  all  the  maximum  estimates.  In  this 
case,  Expert  Ts  maximum  estimate  of  45  deer  is  the  most  pessimistic  of  all  maximum  population  estimates. 
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Hurwicz  Criterion 

The  maximin  and  maximax  criteria  are  frequently  used  when  alternative  actions  are  compared.  The 
maximin  (best  of  the  worst)  and  maximax  (best  of  the  best)  criteria  represent  pessimistic  and  optimistic 
outcomes.  The  Hurwicz  criterion  represents  a  compromise  between  these  two  criteria.  If  we  define  a 
coefficient  of  optimism  as  a  that  ranges  from  0  to  1,  then  1-a  is  a  coefficient  of  pessimism.  By  weighting 
the  maximum  and  minimum  outcomes  by  the  coefficients  of  optimism  and  pessimism,  respectively,  we  can 
obtain  results  such  as  the  following:  Let  a=0.7  reflect  a  view  that  is  more  optimistic  than  pessimistic.  The 
Hurwicz  Criterion  yields  0.3(50)  +  0.7(350)  =  260  deer. 

Other  Criteria 

There  are  other  criteria.  The  minimum,  most  likely,  and  maximum  values  can  be  combined  with  any 
weights  that  reflect  the  decision-makers  belief  in  the  likelihood  of  the  respective  outcomes.  The  dominance 
and  regret  criteria  are  two  other  common  criteria  found  in  the  literature.  There  are  additional  criteria  that 
are  more  complex. 

The  example  presented  here  has  been  kept  simple  to  make  it  easy  to  follow.  The  true  appeal  of  these 
criteria  gets  lost  in  the  simplicity  of  the  example  however.  These  criteria  are  most  useful  when  we  have 
multiple  estimates  of  values  for  numerous  alternatives.  Imagine,  if  you  will,  a  set  of  estimates  of  deer 
population  for  say  eight  alternative  plans.  In  cases  like  this,  these  criteria  become  more  useful  decision 
criteria. 

DECISION  TREES  AND  INFLUENCE  DIAGRAMS 

There  are  many  ways  to  represent  a  decision  problem.  Two  of  them  are  decision  trees,  sometimes 
called  event  trees,  and  influence  diagrams.  An  event  tree  or  decision  tree  is  a  modeling  approach  used  to 
structure  a  decision  problem.  Construction  of  the  tree  helps  to  focus  the  analysts'  attention  on  those  elements 
of  the  decision  problem  that  are  most  important.  The  completed  event  tree  provides  a  blueprint  for 
construction  of  the  computer  model  needed  to  analyze  the  problem. 

The  event  tree  is  not  intended  to  capture  every  element  of  the  decision  problem.  Its  purpose  is  to 
capture  the  most  important  elements  of  the  problem  so  they  can  be  addressed  in  the  analysis.  Thus,  the  goal 
in  developing  a  model  is  to  make  the  analysis  as  simple  as  possible,  but  no  simpler.  Let’s  illustrate  this  with 
a  simple  example  of  an  event  tree  for  a  gated  spillway  that  when  opened  can  cause  manatees  to  be  sucked 
through  the  gates  under  certain  conditions.  Suppose  changes  in  the  maximum  allowable  gate  opening 
(MAGO)  rules  are  being  investigated. 

Figure  5  shows  the  basic  structure  of  the  event  tree  used  to  model  the  situation.  The  analysis  begins 
with  a  random  discharge  event  that  produces  a  head  differential  in  excess  of  that  which  can  be  handled  by 
the  normal  spillway  opening.  The  “bell  curves”  indicate  a  chance  event.  If  the  gate  is  not  opened  there  will 
be  no  manatee  casualties.  There  may  be  flood  damages  but  that  is  not  the  concern  of  this  particular  analysis. 
If  the  gate  is  opened  there  is  some  chance  that  manatees  will  be  affected.  If  they  are  not  there  will  be  no 
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casualties.  If  they  are  affected  there  could  be  injuries  or  death  to  one  or  more  animals.  There  is  uncertainty 
associated  with  each  branching  in  the  event  tree. 

By  assigning  probability  distributions  to  the  uncertain  values  and  constants  to  the  known  values  a 
simulation  model  can  be  built  to  estimate  the  expected  casualties  or  the  probability  of  a  casualty  given  the 
size  of  a  discharge  event.  A  similar  model  could  be  built  for  conditions  with  the  new  MAGO  rules  and  if 
reductions  in  manatee  casualties  result  these  project  outputs  can  be  taken  into  account  in  the  decision 
process.  Decision/event  trees  can  be  a  valuable  tool  for  helping  the  analyst  think  through  the  major  elements 
of  a  problem,  even  if  a  model  is  not  subsequently  built  and  run. 

An  influence  diagram  is  used  to  represent  the  decisions,  uncertainties,  values  and  relationships 
among  these  elements  in  a  decision  problem.  It  is  usually  less  detailed  than  a  decision  tree.  Figure  6  shows 
an  influence  diagram  for  the  above  problem.  The  decision  process  begins  with  an  uncertain  flow  (ovals 
represent  chance  values)  that  effects  the  structure’s  operation.  Structure  operation  influences  manatee 
casualties.  The  basic  elements  of  the  problem  are  here  and  the  influences  are  clear.  There  are  no  feedback 
loops  or  other  values  in  this  simple  model. 
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EXPERT  OPINION 

When  uncertainty  results  because  things  are  unknowable  there  are  few  options.  You  either  ignore 
the  uncertainty  or  you  make  some  best  guesses  about  its  nature.  When  analysis  comes  down  to  guessing, 
we  at  least  want  the  best  guessers  available.  These  are  usually  experts  in  the  domain  of  that  which  we  are 
guessing.  Expert  opinion  is  often  the  best  option  for  estimating  that  which  is  unknown  or  unknowable. 

Professional  Judgment 

There  are  a  lot  of  variables,  relationships,  and  situations  encountered  in  environmental  infrastructure 
planning  that  are  not  only  uncertain  but  that  are  either  unknown  or  unknowable.  How  does  an  ecosystem 
function?  Will  a  species  become  locally  extinct?  When  will  the  next  100-year  flow  occur?  Where  do  you 
get  information  about  those  things  that  are  unknown  or  unknowable?  From  experts.  It  is  the  professional 
judgment  of  the  best  informed  people  on  these  subjects  that  we  generally  must  rely  on. 

Professional  judgment  has  always  been  an  important  part  of  the  Corps’  decision  process.  Often 
professional  judgment  has  been  a  mask  for  that  blind  leap  of  faith  that  must  be  made  in  many  investigations. 
When  data,  information,  theory  and  analysis  have  taken  you  as  far  as  they  can,  it’s  time  to  drag  out  the 
hunches  and  gut  feelings,  dressing  them  up  as  professional  judgment.  To  dispense  with  professional 
judgment  in  such  a  flippant  way  would  be  both  extremely  unfair  and  wrong.  The  knowledge  gained  from 
years  of  experience  and  observation  have  imbued  people  with  an  inherent  understanding  of  systems  and 
situations  that  is  not  easily  reduced  to  a  checklist,  theory,  or  standard  operating  procedures.  Professional 
judgment  is  one  of  the  most  important  tools  available  in  a  risk-based  analysis. 

The  essential  difficulty  with  professional  judgment  often  lies  in  the  identification  of  the  experts 
whose  judgments  will  be  sought.  There  is  considerable  professional  literature  that  suggests  that  so-called 
experts  often  do  not  know  the  limits  of  their  own  knowledge  and  expertise.  It  is  common  procedure  for 
experts  to  overstate  their  confidence.  In  other  words,  the  experts  aren’t  right  as  often  as  they  think  they  are. 

It  is  important  to  systematically  search  for  the  expertise  that  is  required.  The  most  knowledgeable 
person  in  the  office  may  not  be  sufficiently  expert  on  a  subject  to  provide  a  full  range  of  values  or  impacts 
that  could  result  from  an  uncertain  situation.  The  consultants  with  whom  you  have  a  contract  may  be  willing 
to  do  the  work,  but  they  may  lack  true  expertise  in  an  area.  It’s  a  good  general  rule  to  search  for  and  verify 
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the  expertise  of  your  experts.  Using  multiple  experts  can  help  assure  a  more  complete  picture  of  the 
uncertainty  involved  in  a  situation. 

Elicitation/Opinion  Analysis 

Experts  have  considerable  knowledge.  Obtaining  it  from  them  in  a  useful  and  reliable  manner  is  not 
always  as  easy  as  it  might  seem.  Suppose  for  example  there  is  concern  that  some  species  of  interest  will 
become  locally  extinct.  Take  something  as  simple  as  the  beaver.  How  would  you  go  about  getting  an  expert 
to  estimate  this  event?  How  do  we  define  local?  Extinct?  Over  what  time  frame? 

The  process  of  discovering  and  quantifying  the  knowledge  and  opinions  of  experts  so  they  can  be 
communicated  to  others  is  called  elicitation  or  opinion  analysis.  In  risk-based  analysis  both  the  likelihood 
and  the  consequences  of  uncertain  events  can  be  unknown  or  unknowable.  Environmental  planning,  because 
of  its  complexity  and  its  relative  newness,  is  full  of  unknown  and  unknowable  events  and  situations. 
Eliciting  information  from  experts  will  likely  be  an  extremely  important  analytical  method  for  risk  analysts 
involved  in  this  area. 

Subjective  Probability  Assessment 

Estimating  the  likelihood  of  events  is  a  special  subfield  of  elicitation  that  has  received  considerable 
attention  in  the  human  behavior  and  decision-making  literature.  Subjective  probability  assessments  are 
techniques  used  to  elicit  from  experts  the  probabilities  of  various  events.  Subjective  probabilities  (see 
section  on  Probability)  are  not  based  on  analytical  or  frequency-based  probability  estimates.  They  are 
opinions  based  on  the  expert’s  strength  of  belief 

Experts  assess  and  evaluate  the  available  evidence  and  then  use  their  professional  judgment  to 
estimate  the  likelihood  of  various  events.  These  likelihoods  can  be  point  estimates  (e.g.,  there  is  a  0.2  chance 
beaver  will  be  extinct  within  30  years)  or  estimates  of  entire  distributions  (there  is  a  0.9  chance  that  there 
will  be  100  or  less  beavers  per  acre,  a  0.6  chance  there  will  be  45  or  fewer  beavers,  a  0.2  chance  of  1 5  or 
fewer  beavers  and  0.01  chance  of  3  or  fewer  beavers).  There  is  no  right  or  wrong  answer,  only  opinions  that 
are  more  or  less  accurate. 

Analytic  Hierarchy  Process 

The  analytic  hierarchy  process  (AHP)  was  initially  developed  as  a  multi-criteria  decision  tool. 
Alternatives  are  chosen  on  the  basis  of  their  contributions  to  various  objectives.  AHP  is  a  tool  that  can  be 
used  to  make  decisions  in  situations  involving  multiple  objectives.  Consider  a  decision  problem  like  that 
shown  in  Figure  7  where  we  have  a  goal  (the  top  square),  criteria  for  reaching  that  goal  (the  second  row  of 
squares),  and  alternatives  for  meeting  each  criterion  (the  tic  marks  beneath  the  criteria).  The  number  of 
levels  is  purely  arbitrary  with  this  technique;  there  can  be  as  many  levels  of  sub-criteria  as  desired. 

Using  pairwise  comparisons,  first  of  the  criteria  (and  sub-criteria),  then  of  the  alternatives  for 
reaching  each  criterion,  this  technique  produces  index  numbers  for  each  alternative  plan  that  reflect  the 


41 


An  Introduction  to  Risk  and  Uncertainty 
in  the  Evaluation  of  Environmental  Investments 


collective  judgments  of  the  decision-maker/expert.  These  index  numbers  identify  the  best  alternative 
consistent  with  the  experts  Judgment. 

The  pairwise  comparisons  proceed  at  each  level  of  the  hierarchical  structure.  For  example,  food 
source  is  compared  first  to  cover,  then  food  source  is  compared  to  the  disturbance  factor,  and  finally  it  is 
compared  to  reproduction  potential.  At  each  comparison  the  expert  may  assign  a  numerical  or  qualitative 
rating  of  the  two  factors,  e.g.,  food  =  1,  cover  =  3  or  cover  is  moderately  more  important  than  food.  Each 
criterion  is  eventually  compared  to  every  other  criterion.  Then  each  alternative  is  compared  to  every  other 
alternative  for  each  criterion.  For  example,  with  regard  to  food  sources  alternative  1  is  strongly  more 
important  than  alternative  2.  With  each  such  judgment  the  index  number  is  constructed  reflecting  the 
expert’s  relative  weighting  of  each  criteria  and  alternative. 

This  technique  can  be  readily  adapted  to  situations  involving  uncertainty.  In  this  case,  the  weights 
of  sub-criteria  and  criteria  are  not  the  relative  importance  of  these  items  but  their  relative  likelihoods.  AHP 
has  proven  to  be  a  useful  tool  in  quantifying  the  subjective  judgments  of  experts  in  a  variety  of  decisions 
under  uncertainty.  One  of  the  more  recent  applications  of  this  technique  was  the  development  of  structure 
condition  indices  for  the  80  largest  locks  and  dams  in  the  Corps’  inland  waterway  system. 

PROBABILITY 

Many  situations  of  uncertainty  are  represented  by  probabilities.  An  event  such  as  a  drought  or 
extreme  flow,  for  example,  may  or  may  not  happen.  What  is  the  probability  it  will  occur  this  year?  A 
sample  survey  may  indicate  an  indigenous  population  of  white  tail  deer  of  850,  with  95%  certainty  the  true 
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population  is  between  700  and  1,000  deer.  We 
might  be  interested  in  the  likelihood  that  beavers 
will  become  locally  extinct  in  the  absence  of  a 
project.  Experts  would  be  relied  upon  to  estimate 
this  probability  subjectively.  There  are  endless 
possible  uses  for  probabilities  in  environmental 
planning. 

Corps  personnel  doing  risk  analysis  have 
to  be  extremely  comfortable  with  the  basic  notions 
of  probability  and  probability  distributions.  They 
need  to  understand  the  differences  among 
analytical,  empirical/frequentist,  and  subjective 
notions  of  probabilities.  The  laws  and  language  of 
probability  are  minimum  requirements  for  risk 
analysts. 

A  basic  knowledge  of  the  most  common  discrete  and  continuous  distributions  and  when  to  use  each 
is  essential.  This  knowledge  should  include  a  knowledge  of  the  distribution  parameters,  their  shapes  and 
locations,  as  well  as  distribution-fitting  methods.  Analysts  should  be  able  to  move  back  and  forth  between 
probability  density  functions  and  the  cumulative  distribution  functions. 

Armed  with  these  skills,  Corps  personnel  will  have  the  probability  background  necessary  for 
understanding  the  chance  dimension  of  uncertain  situations.  Furthermore,  this  knowledge  is  a  necessary 
precursor  to  the  use  of  more  sophisticated  models,  methods,  and  techniques. 

PROBABILITY  DISTRIBUTIONS 

When  the  Corps  first  began  to  incorporate  risk  analysis  into  its  planning  studies  a  fundamental  goal 
was  to  get  analysts  to  represent  the  value  of  critical  variables  as  distributions  rather  than  as  determinant 
single  values.  Issues  of  which  distribution  to  use  were  clearly  secondary  to  taking  the  first  step  and  using 
a  distribution.  Now  that  risk  analysis  is  gaining  more  widespread  acceptance  analysts  readily  embrace  the 
use  of  probability  distributions.  The  question  has  now  become,  which  distribution  should  I  use?  For 
example,  what  distribution  should  we  use  to  represent  the  uncertainty  attending  the  HSI  index? 

Suppose  we  could  measure  every  striped  bass  in  the  bay.  What  would  a  graph  of  these  data  look 
like?  A  histogram  is  a  bar  graph  that  depicts  the  frequency  with  which  values  occur  in  a  data  set.  Figure  8 
presents  a  histogram  of  fish  sizes  that  range  from  about  2  cm  to  over  180  cm.  Most  of  the  fish  fall  in  the 
range  of  90  cm.  Each  bar  is  10  cm  wide.  If  each  bar  were  only  1  cm  wide  the  graph  would  take  on  the 
smooth  look  of  a  normal  distribution  like  the  one  shown  in  Figure  9.  Histograms  are  pictures  of  frequency 
distributions.  They  show  how  values  in  a  data  set  are  distributed. 


-  ^ 

Three  Approaches  to  Determining  A  Probability 

The  three  approaches  to  determining  a 
probability  are  classical  probability,  empirical 
probability,  and  subjective  probability.  In  classical 
probabilities  all  outcomes  are  equally  likely. 

Examples  include  the  probability  of  rolling  a  six  with 
a  die,  or  drawing  a  king  from  a  deck  of  cards. 

Empirical  probabilities  are  based  on  past  experience. 
Most  hydrology  is  based  on  empirical  probabilities, 
i.e.,  the  observed  frequencies  of  events. 

Probabilities  based  on  available  evidence  and  an 
individual’s  own  judgment  are  called  subjective 
probabilities. 
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Histograms  need  not  have  the  neat  symmetry  of  Figure  9.  They  can  assume  any  shape  at  all.  A  histogram 
is  a  picture  of  how  the  elements  of  a  data  set  are  distributed.  A  probability  distribution  is  a  picture  of  how 
all  the  elements  of  a  population  is  distributed.  The  distinction  between  the  two  is  that  a  histogram  is  usually 
a  picture  of  some  of  the  data  and  a  probability  distribution  is  a  picture  of  all  the  data.  Because  having  “all 
the  data”  is  often  a  practical  impossibility,  probability  distributions  are  often  theoretical  rather  than 
empirically  observed  the  way  a  histogram  is. 
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For  example,  the  histogram  above  was  presented  as  a  picture  of  the  length  of  all  the  fish  in  the  bay. 
Analysts  never  have  that  kind  of  information.  The  histogram  above  would  more  likely  be  a  picture  of  the 
distribution  of  lengths  for  1,000  fish  sampled  from  the  bay.  Nonetheless  there  is  a  theoretical  distribution 
of  the  length  of  all  striped  bass  living,  dead  or  yet  to  be  bom  and  it’s  called  the  probability  distribution.  That 
distribution  yields  information  like  the  probability  that  a  fish  will  be  76.4  cm  or  more  in  length  and  that 
probability  can  be  estimated  from  the  theoretical  probability  distribution  of  fish  lengths. 

The  histogram  yields  information  about  the  frequency  with  which  fish  76.4  cm  or  longer  are 
observed  in  the  sample  of  1,000  fish.  As  the  sample  gets  bigger  and  bigger  the  frequency  with  which  fish 
76.4  cm  or  more  are  observed  gets  closer  and  closer  to  the  true  probability  with  which  that  occurs. 
Frequency  distributions  or  histograms,  then,  are  approximations  of  probability  distributions. 

Analysts  are  interested  in  more  than  fish  lengths.  They  may  be  interested  in  the  distribution  of 
stream  discharges,  tidal  fluctuations,  salinities,  weights  of  animals,  heights  of  plants,  depths  of  water,  speed 
of  winds,  and  so  on.  Each  of  these  characteristics  has  a  parent  population  that  has  some  probability 
distribution.  Experience  and  theory  have  revealed  that  these  distributions  tend  to  assume  certain  basic 
shapes;  or,  at  least,  they  have  certain  basic  characteristics. 

The  shapes  of  many  of  these  distributions  can  be  described  by  a  mathematical  formula.  The  normal 
distribution,  for  example,  is  described  by  the  following  formula: 

Ax)=— !—e 
(27t:)-^o 


where  f(x)  is  the  probability  density  function,  what  we  have  been  calling  the  probability  distribution.  The 
values  of  tz  and  e  are  known.  Unknown  are  the  values  of  p  and  o.  These  are  the  parameters  or  constants 
which,  once  known,  give  the  distribution  its  location  along  a  real  number  line  and  its  shape  above  that  line. 
The  population  mean  is  p,  the  Greek  mu,  and  o,  sigma,  is  the  standard  deviation  of  the  population.  Hence, 
given  the  population  mean  and  standard  deviation  any  normal  distribution  can  be  described.  The  normal 
distribution  in  Figure  9  is  centered  about  a  mean  of  90  and  its  spread  is  dictated  by  its  standard  deviation  of 
30. 


Other  distribution  shapes  are  described  by  other  formulas.  Frequently,  the  formulas  describe 
families  of  distributions.  That  is,  when  the  parameters  of  the  distribution  change  the  shape  and  location  of 
the  distribution  can  change  as  well.  For  example,  there  is  not  one  normal  distribution"  but  an  entire  family 
of  distributions.  Even  though  each  has  that  distinctive  single-peaked  symmetrical  bell  shape  each 


^  ^  Although  the  normal  distribution  is  unique  in  that  any  normal  distribution  can  be  converted  to  the  standard  normal  distribution, 
also  known  as  the  Z  distribution. 
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distribution  can  vary  in  its  location  on  the  normal  line  and  the  extent  to  which  it  is  tall  and  thin  or  short  and 
fat.  Other  families  of  distributions  can  vary  in  far  more  dramatic  ways. 

What  are  we  to  assume  about  the  nature  of  the  probability  distributions  that  are  used  to  describe  the 
uncertainty  in,  for  example,  environmental  restoration  studies?  If  empirical  data  are  available,  visual 
assessments  of  the  frequency  distribution  may  be  used  to  approximate  the  underlying  probability  distribution. 
Statistical  distribution  fitting  tests  using  goodness-of-fit  measures  can  be  used  to  more  effectively  test  how 
well  the  data  conform  to  any  given  distribution.  The  chi-square  and  Kolmogorov-Smirnov  statistics  are 
goodness-of-fit  measures  most  often  used  to  determine  if  one’s  data  are  consistent  with  a  normal  distribution 
or  any  other  distribution.  These  tests  are  available  from  a  number  of  commercially  available  statistics 
packages.  BestFit  from  Palisades  Corporation  is  especially  designed  for  distribution  fitting  tests. 

When  there  are  no  empirical  data  to  examine,  theory  or  understanding  the  process  that  determines 
the  data  of  interest  may  suggest  the  distribution  to  use.  Prior  studies  and  the  literature  may  offer  insights  into 
the  nature  of  the  distributions  that  are  appropriate  to  use.  The  books  A  Guide  to  Probability  Theory  and 
Application  by  C.  Derman,  L.  Glaser  and  I.  Olkin  (1973)  and  Uncertainty:  A  Guide  to  Dealing  with 
Uncertainty  in  Quantitative  Risk  and  Policy  Analysis  by  M.  Granger  Morgan  and  Max  Henrion  (1990)  are 
two  sources  of  such  insight.  The  users’  manuals  for  @RISK  and  Crystal  Ball  are  two  examples  of 
commercial  software  products  that  provide  some  useful  information  about  probability  distributions. 

In  the  absence  of  better  information,  a  systematic  process  of  selecting  a  distribution  may  be  the  best 
approach  in  many  circumstances.  Following  these  few  simple  steps  may  help  the  analyst  make  a  reasonable 
choice  of  a  distribution  for  representing  the  uncertainty  when  there  is  no  better  guidance  on  distribution 
choice.  First,  clearly  identify  the  variable  of  interest.  Second,  list  everything  you  know  about  the  variable 
being  sure  to  include  quantitative  and  qualitative  aspects  of  the  variable.  Third,  identify  the  type(s)  of 
uncertainty  with  which  you  are  dealing.  Fourth,  review  the  approaches  to  the  different  types  of  uncertainty. 
Fifth,  list  the  description  and  characteristics  of  the  candidate  probability  distributions.  Sixth,  identify  the 
distribution(s)  that  best  characterizes  the  variable  in  question.  If  none  of  these  methods  identify  a  single  best 
distribution  use  sensitivity  analysis  to  determine  the  significance  of  choosing  one  distribution  over  another. 

The  remainder  of  this  section  attempts  to  acquaint  the  reader  with  some  of  the  most  commonly 
encountered  families  of  distributions  used  by  risk  analysts.  It  is  impossible  to  do  justice  to  this  subject 
matter  in  a  few  sentences.  Undoubtedly,  it  would  be  interesting  to  see  the  formulas,  discuss  the  parameters, 
and  look  at  sample  distributions.  The  subject  matter  is  too  complex  to  do  that  here.  Instead  the  briefest  of 
introductions  to  the  distributions  is  offered.  Reasonably  accessible  additional  detail  can  be  obtained  from 
the  books  cited  above.  A  more  concise  and  mathematical  description  can  be  found  in  Statistical 
Distributions  by  Merran  Evans,  Nicholas  Hastings,  and  Brian  Peacock  or  in  many  advanced  statistics  texts. 
In  addition,  there  are  several  software  packages  that  can  be  used  to  explore  these  distributions.  RISKview 
and  eHOP'^  are  two  packages  that  allow  the  user  to  explore  the  characteristics  of  a  variety  of  distributions. 


RISKview  is  available  from  Palisades  Corporation  and  eHOP,  The  Eleetronic  Handbook  of  Probability,  is  available  from  the 
Crunch  Software  Corporation  at  415-584-0122. 
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Normal  Distribution 

The  normal  distribution  is  probably  the  most  common  distribution  encountered.  Variables  formed 
by  the  addition  of  other  uncertain  quantities  are,  generally,  normally  distributed.  A  total  cost  that  is  the  sum 
of  several  uncertain  line  item  costs  will  tend  to  be  normally  distributed.  Uncertainty  that  results  from 
unbiased  measurement  errors  can  be  represented  by  a  normal  distribution.  Thus,  if  the  chief  concern  about 
the  size  of  a  local  population  of  a  species  is  the  potential  measurement  error,  the  uncertainty  may  be 
represented  by  a  normal  distribution.  Many  other  distributions  approach  or  reduce  to  a  normal  distribution 
under  circumstances  (i.e.,  for  certain  parameter  distributions)  that  vary  from  distribution-to-distribution.  The 
Central  Limit  Theorem  contributes  to  the  utility  of  the  normal  distribution  in  assuring  that  the  distribution 
of  some  sample  statistics  is  always  normal,  regardless  of  the  parent  distribution  from  which  the  sample  was 
drawn,  as  long  as  the  sample  size  is  large  enough. 

The  normal  distribution  can  be  described  by  two  parameters,  the  mean  and  standard  deviation.  For 
quantities  where  negative  values  are  not  meaningful  (populations,  costs,  flows)  it  may  be  inappropriate  to 
use  the  normal  distribution  unless  it  is  truncated  to  eliminate  the  illogical  values.  Distributions  can  be 
truncated  at  any  values  required  by  the  analysis.  Figure  1 0  shows  Figure  9  truncated  at  values  of  20  and  1 50. 


V 

Figure  10:  Truncated  Normal  Distribution 
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Lognormal  Distribution 

If  the  logarithm  of  the  random  variable  of  interest  is  normally  distributed  the  random  variable  has 
a  lognormal  distribution.  Variables  formed  by  the  multiplication  of  other  uncertain  quantities  tend  to  have 
a  lognormal  distribution  because  the  sum  of  the  logs  approaches  the  normal  distribution.  For  example,  an 
uncertain  area  estimate  times  an  uncertain  habitat  suitability  index  will  yield  a  lognormal  distributed  estimate 
of  the  number  of  habitat  units. 

If  we  know  a  value  within  an  order-of-magnitude,  say  a  factor  of  two  or  ten  and  when  uncertainties 
are  large,  the  lognormal  is  appropriate  to  use.  The  lognormal  distribution  cannot  be  used  with  non-positive 
values  so  it’s  useful  when  negative  values  of  the  variable  of  interest  are  not  permitted.  It  is  a  useful 
distribution  to  use  when  values  are  positively  skewed,  i.e.,  most  values  occur  near  the  minimum  value.  The 
distribution  is  described  by  the  mean  of  the  logs  of  the  random  variable  and  the  standard  deviation  of  the  log 
of  the  variable. 

Exponential  Distribution 

The  time  between  events  that  are  purely  random'^  often  has  an  exponential  distribution  as  does  the 
size  of  random  events.  The  exponential  distribution  can  be  used  to  describe  the  duration  and  size  of  events, 
like  storms  and  oil  spills.  It  is  commonly  used  in  reliability  theory,  waiting  times  and  queuing  theory.  It  is 
described  by  a  single  parameter,  k,  that  is  equal  to  one  divided  by  the  average  time  between  events.  Thus, 
the  single  parameter  is  the  rate  at  which  the  events  occur'". 

The  exponential  distribution  has  a  mode  of  zero  and  is  constrained  to  non-negative  numbers. 
Because  the  distribution  has  a  fixed  shape  the  single  parameter  determines  only  the  scale  of  the  distribution. 
It  is  not  a  very  flexible  distribution  and  is  useful  for  a  very  limited  type  of  phenomena. 

Poisson  Distribution 

Unlike  the  eontinuous  distributions  above,  the  Poisson  distribution  is  a  discrete  probability 
distribution  that  is  constrained  to  non-negative  numbers.  The  Poisson  is  useful  when  we're  dealing  with  the 
number  of  oceurrences  of  an  event  over  a  specific  interval  of  time  or  space.  For  example,  the  number  of 
plants  per  acre  or  the  number  of  floods  per  100  years  may  have  Poisson  distributions.  If  a  single  trial  can 
be  defined  as  some  fixed  area  of  opportunity,  like  a  fixed  length  of  time  (minute,  day,  year,  planning  period) 
or  space  (per  foot,  acre,  cubic  meter)  the  Poisson  may  be  useful.  The  number  of  events  that  occur  in  the 
fixed  time  period  or  space  may  have  a  Poisson  distribution. 


^  A  purely  random  process  of  events  is  called  a  Poisson  process.  The  exponential,  Poisson,  and  gamma  distributions  deal  with 
Poisson  processes. 

Suppose  the  average  time  between  customers  is  ten  minutes.  The  parameter  is  0.1,  and  O.I  is  the  number  of  customers  per 

minute. 
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The  single  parameter  of  the  distribution,  a,  is  the  average  number  of  events  per  area  of  opportunity, 
e.g.,  the  average  number  of  floods  per  1 00  years  or  the  average  number  of  plants  per  acre.'’  Like  other  single 
parameter  distributions,  the  Poisson  lacks  flexibility.  When  the  parameter  is  large  the  discrete  Poisson 
distribution  may  be  approximated  by  a  continuous  normal  distribution. 

Gamma  Distribution 

Poisson  processes  confined  to  non-negative  values  that,  for  various  reasons,  yield  distributions 
skewed  to  the  right  can  be  described  by  a  gamma  distribution.  The  bulk  of  the  area  under  a  gamma  density 
function  is  located  near  the  origin  and  the  density  function  area  drops  gradually  as  the  random  variable 
increases  in  value.  The  gamma  distribution  describes  the  time  required  for  k  events  to  occur. 

The  gamma  distribution  is  a  two  parameter  distribution  that  describes  a  Poisson  process.  The  shape 
of  the  distribution  is  determined  by  k,  while  X  is  the  scale  parameter.  This  distribution  is  similar  to  the 
lognormal  but  it  is  less  positively  skewed  and  less  tail-heavy.  Thus,  it  may  be  used  in  situations  similar  to 
those  appropriate  to  the  lognormal.  The  gamma  is  useful  for  describing  times  (or  spaces)  between  events 
(items)  that  are  not  pure  random  processes.  Some  quantities  frequently  exhibiting  skewed  distributions  like 
the  gamma  include  physical  quantities  such  as  precipitation  quantities  and  pollutant  concentrations  as  well 
as  the  time  between  malfunctions  of  aircraft  engines,  time  between  customers  at  a  checkout  counter,  and  the 
time  required  to  complete  maintenance  on  an  engine. 

WeibuII  Distribution 

The  Weibull  distribution  is  actually  a  family  of  distributions  that  can  assume  the  properties  of  several 
other  distributions.  With  location,  scale,  and  shape  parameters  the  Weibull  can  resemble  the  exponential 
and  normal  distributions  as  well  as  assume  a  number  of  other  shapes. 

The  performance  of  many  complex  systems  depends  upon  the  reliability  of  the  components  making 
up  the  system,  perhaps  an  ecosystem.  The  time  to  failure  or  length  of  life  of  a  component  from  a  specified 
time  to  its  failure  can  often  be  described  by  a  Weibull  distribution.  It  has  also  been  used  to  describe  some 
physical  quantities  like  wind  speeds.  In  its  more  common  form  it  is  similar  in  shape  to  the  gamma 
distribution  though  it  is  less  skewed  and  tail  heavy.  Use  it  only  with  non-negative  values. 

Beta  Distribution 

The  beta  distribution  yields  a  two  parameter  distribution  that  is  defined  over  the  interval  from  zero 
to  one.  There  is  a  four-parameter  form  of  the  beta  distribution  where  the  additional  parameters  define  the 
range  endpoints.”  It  is  a  very  flexible  distribution  for  representing  variability  over  a  fixed  range.  Defining 


Alpha,  a,  is  defined  as  the  average  number  of  events  per  time  (or  space)  interval,  k,  times  the  length  (size)  of  the  time  (space) 
interval,  T.  Thus  we  define,  a  =  A.T. 


16 


The  additional  parameters  translate  the  distribution  and  change  its  scale,  so  it  is  possible  to  model  variation  over  any  scale. 
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the  way  a  variable  varies  over  a  range  makes  the  beta  a  valuable  distribution  for  modeling  proportions, 
fractions  and  percentages.  It  can  be  used  to  represent  the  probability  of  occurrence  of  an  event  or  the 
proportion  of  time  that  a  structure  or  system  functions  as  designed. 

Uniform  Distribution 

The  simplest  of  all  probability  distributions  is  the  uniform  distribution,  where  the  random  variable 
assumes  all  its  values  with  equal  probability.  The  probability  that  a  value  from  a  given  sub-interval  will  be 
realized  only  depends  on  the  size  of  the  sub-interval  relative  to  the  overall  interval.” 

Use  the  uniform  distribution  when  you  can  identify  a  range  of  possible  values  but  are  unable  to  say 
which  values  in  that  range  are  more  likely  to  oceur.  In  other  words,  the  uniform  ean  be  used  when  any  one 
value  is  as  likely  as  any  other.  It  is  a  two-parameter  distribution  where  the  parameters  are  the  minimum  and 
maximum  values  that  can  occur.  Subjective  reasoning  can  often  be  used  to  estimate  the  parameters.  When 
the  uncertainties  are  large  and  somewhat  asymmetric  you  may  use  a  loguniform  distribution  to  make  them 
more  orderly.'* 

Triangular  Distribution 

Most  likely  values  are  frequently  identified  in  the  planning  process.  If  it  is  believed  that  values  close 
to  the  most  likely  value  will  occur  more  often  than  values  near  the  extreme  ends  of  the  range  of  possible 
values  a  triangular  distribution  may  be  used.  The  arbitrary  shape  of  the  triangular  distribution  and  its  sharp 
comers  telegraph  the  fact  that  the  precise  nature  of  the  distribution  are  unknown.  A  default  distribution  of 
sorts,  the  triangular  should  not  be  used  too  quickly  or  too  frequently.  Its  three  parameters  are  the  most  likely, 
minimum  and  maximum  values  the  variable  can  assume.  If  the  uncertainties  are  large,  a  logtriangular 
distribution  can  be  used. 

Binomial  Distribution 

The  binomial  distribution  gives  the  probability  of  obtaining  x  events,  each  with  probability  p  of 
occurring,  from  n  independent  trials  where  each  trial  has  only  two  outcomes,  frequently  called  success  and 
failure.  We  might  want  to  know  the  probability  of  observing  two  or  more  terns  with  toxic  levels  of  cadmium 
from  a  group  of  100  terns  when  the  probability  that  a  tern  has  a  toxic  level  is  0.005.  Its  parameters  are  the 
number  of  events,  x,  and  the  probability  of  the  event  occurring  on  a  given  trial,  p.  Use  the  binomial 
distribution  when  the  probability  of  success,  p,  or  failure,  1  -  p,  are  known.  Do  not  use  the  binomial 
distribution  when  the  event  probability  is  unknown. 


If  the  bus  always  comes  between  7:00  and  7:10,  the  probability  that  it  will  come  between  7:05  and  7:07  is  exactly  the  same  as 
the  probability  that  it  will  come  between  7:02  and  7:04. 

1 

For  a  random  variable  X  create  Y  =  InX.  The  loguniform  is  a  uniform  distribution  of  Y. 
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Discrete  Distribution 

A  discrete  distribution  allows  for  a  finite  number  of  discrete  outcomes.  Each  possible  outcome  is 
assigned  a  probability.  The  probabilities  must  be  mutually  exclusive  and  sum  to  one.  For  example,  when 
describing  the  performance  of  a  tidal  wetland  restoration  project  we  would  identify  three  discrete  outcomes: 
1)  tidal  wetlands,  2)  mudflats,  and  3)  daisies.  We  might  estimate  the  probabilities  of  these  events  as  0.6,  0.2, 
and  0.2,  respectively. 

Summary  of  Distribution  Usage 

The  proper  distribution  to  use  is  an  empirical  issue.  Ideally,  we'd  like  to  fit  a  distribution  to  some 
existing  data  to  determine  the  distribution  to  use.  Lacking  data,  theory  and  similar  studies  can  be  used  for 
suggestions.  Undoubtedly,  this  will  frustrate  many  people  because  it  doesn’t  provide  specific  guidance  on 
which  distributions  to  use  and  when  to  use  them.  The  truth  of  the  matter  is  there  is  much  yet  to  be  learned 
about  the  uncertainty  confronted  in  environmental  infrastructure  studies.  Table  4  summarizes  the 
distributions  described  above. 

RANDOM  SAMPLING 

Information  from  a  sample  used  to  make  generalizations  about  the  population  from  which  it  came 
must  be  representative  of  that  population.  In  order  for  a  sample  to  be  representative  of  the  population,  the 
elements  of  the  sample  should  be  randomly  selected  from  the  population.  The  basic  idea  here  is  that  if  every 
member  of  the  population  has  the  same  possibility  of  being  selected  as  part  of  the  sample  and  if  the  sample 
size  is  large  enough,  pure  chance  will  generally  assure  a  representative  sample. 

Simple  random  sampling  is  the  most  common  method  for  obtaining  such  a  sample.  Stratified 
random  sampling,  cluster  sampling,  two-stage  cluster  sampling,  sequential  sampling,  and  systematic 
sampling  are  sampling  techniques  with  which  those  in  environmental  planning  should  be  familiar.  Quadrat 
sampling  involves  estimating  the  number  or  percentage  of  elements  (deer,  fish,  trees,  submerged  substrate, 
people,  etc.)  in  a  defined  area  or  quadrat  (per  acre,  per  hectare).  Quadrat  sampling  is  useful  in  estimating 
population  densities  and  size. 

It’s  not  practical  to  expect  all  Corps  analysts  to  be  expert  in  sampling  techniques.  The  techniques 
are  quite  straightforward  and  not  at  all  difficult  to  implement.  Many  analysts  will  be  able  to  master  them 
with  modest  training  and  support.  Inevitably,  complex  sampling  situations  will  arise  in  which  it  will  be 
sufficient  for  Corps  analysts  to  be  familiar  with  the  existence  of  other  tools  and  methods  so  they  can 
supervise  and  review  the  work  of  others. 

SENSITIVITY  ANALYSIS 

This  is  one  of  the  more  practical  methods  for  introducing  risk  and  uncertainty  analysis  to 
environmental  restoration  projects.  Completing  an  analysis  of  an  uncertain  situation  requires  numerous 
assumptions.  It  might  be  important  to  know  if  the  results  of  the  analysis  are  sensitive  to  the  assumptions  that 
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have  been  made.  An  investigation  that  is  intended  to  determine  the  sensitivity  of  the  analysis’  results  to 
different  assumptions  is  called  a  sensitivity  analysis.  It  is  also  called  parametric  variation.  A  number  or 
parameter  that  is  treated  like  a  constant  in  the  analysis  is  varied  systematically  over  a  range  of  values  to 
examine  its  impact  on  the  decision. 


Table  4:  Distribution  Summary 

Normal  distribution 

•  Variable  has  a  most  likely  value  (mean) 

•  Uncertain  value  as  likely  above  as  below  mean 

•  Uncertain  value  near  mean  more  likely  than  far  away 

Lognormal  distribution 

•  Variable  is  non-negative 

•  Uncertain  value  more  likely  near  lower  limit 

•  Natural  log  of  variable  yields  normal  distribution 

Exponential  distribution 

•  Describes  duration  and  size  of  events 

•  Variable  is  non-negative 

Poisson  distribution 

•  Single  parameter  discrete  distribution  confined  to  non-negative  values 

•  Number  of  events  over  interval  of  time  or  space 

Gamma  distribution 

•Time/space  between  events/items 
•  Two  parameter  distribution  of  non-negative  variable 

Weibull  distribution 

•  Three  parameter  distribution  of  non-negative  variable 

•  Can  resemble  many  other  distributions 

•  Time-to-failure,  life  of  component 

Beta  distribution 

•  Two  or  four  parameter  distribution 

•  Defines  variation  over  a  range 
•Proportions,  fractions,  and  percentages 

Uniform  distribution 

•  Minimum  value  is  fixed 

•  Maximum  value  is  fixed 

•  All  values  between  min  and  max  are  equally  likely 

Triangular  distribution 

•  Minimum  value  is  fixed 

•  Maximum  value  is  fixed 

•  Most  likely  value  falls  between  min  and  max,  with  values  near  min  and  max  less 
likely  than  values  near  most  likely 

Binomial  distribution 

•  Only  two  outcomes  possible 

•  Trials  are  independent 

•  Probability  of  “x”  events  in  “n”  tries  given  probability  on  any  trial  is  “p” 

Discrete  distribution 

•  Finite  integer  number  of  outcomes  possible 

•  Each  outcome  has  independent  probability  of  occurrence 
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Consider  a  very  simple  sensitivity  analysis/parametric  variation.  Suppose  you  have  1 00  acres  of  land 
and  you  estimate  the  HSI  to  be  0.5.  There  are  50  HU’s  in  this  example.  But,  suppose  the  HSI,  normally 
treated  like  a  constant,  was  a  matter  of  professional  judgment  that  was  uncertain.  Suppose  the  HSI  could 
be  0.75.  Then  we  have  75  HU’s.  The  number  of  HU’s  is  very  sensitive  to  the  HSI  estimate.  The  HSI  was 
the  parameter  in  this  example  that  was  allowed  to  vary. 

The  purpose  of  a  sensitivity  analysis  is  at  least  two-fold.  Sensitivity  analyses  conducted  early  in  the 
investigation  can  help  you  identify  the  key  parameters/variables/factors.  If  you  vaiy  one  of  these  values  and 
the  result  changes  significantly,  you  know  you  have  identified  an  important  source  of  uncertainty  or  a  key 
parameter.  In  this  case  you  do  one  of  two  things.  First,  you  should  try  to  reduce  the  uncertainty  if  at  all 
possible.  Second,  if  the  uncertainty  cannot  be  reduced  it  must  be  addressed'^  and  its  potential  importance 
in  the  decision-making  process  to  the  decision-makers  needs  to  be  explained. 

A  second  purpose  of  sensitivity  analysis  may  be  to  examine  the  cumulative  effects  of  various 
uncertain  factors.  For  example,  suppose  the  area  estimate  and  the  HSI  are  both  uncertain.  Though  neither 
alone  may  have  a  significant  impact  on  the  HU’s,  the  two  together  might. 

Sensitivity  analysis  is  a  controlled  and  deliberate  process.  Allowing  things  to  vary  randomly  over 
a  range  of  values,  as  one  does  in  a  Monte  Carlo  process,  is  not  sensitivity  analysis.  It  would  not  be  proper, 
for  example,  to  do  a  simulation  in  which  the  period  of  analysis  is  allowed  to  vary  according  to  some 
probability  distribution.  We  may  be  unsure  how  long  a  project  will  last  but  that  is  not  a  value  one  allows 
to  vary  randomly.  The  proper  approach  is  to  vary  that  parameter  systematically.  Consider  the  plan  impacts 
at  25, 30,  50  or  whatever  number  of  years  is  reasonable  and  see  if  the  decision  would  vary  with  the  period 
of  analysis. 

Sensitivity  analysis  is  a  valuable  tool.  It  can  be  a  reasonable  compromise  between  a  detailed  and 
sophisticated  risk-based  analysis  and  completely  ignoring  the  uncertainty  that  exists  in  your  investigations. 
Identifying  the  important  variables  in  an  investigation  and  varying  the  key  parameters  and  noting  the  effect 
on  the  best  decision  can  be  an  effective  way  to  begin  to  address  risk  and  uncertainty  in  environmental 
projects. 

SIMULATIONS 

Simulation  is  a  technique  used  to  answer  what-if  types  of  questions.  It  is  used  most  appropriately 
when  a  decision  problem  under  analysis  is  too  complex  to  be  solved  by  analytical  models.  Simulation  is  a 
quantitative  procedure  that  describes  a  process  by  constructing  a  model  of  the  process  and  then  observing 
how  the  model  behaves  over  a  series  of  iterations  in  order  to  learn  how  the  process  itself  might  behave. 


In  some  instances  it  may  be  appropriate  to  treat  these  critical  variables  in  a  probabilistic  fashion.  In  others,  they  will  be  varied  in 
a  systematic  fashion.  The  determining  faetors  are  the  types  of  quantities  that  are  uncertain  and  the  nature  of  the  uncertainty.  See  Chapter  Four 
for  additional  discussion  of  this  point. 
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The  simulations  that  Corps  personnel  use  are  most  often  mathematical  models  that  cannot  be  solved 
analytically  so  they  are  run  on  trial  data  to  simulate  the  behavior  of  a  system.  As  a  general  rule,  simulation 
should  be  used  only  as  a  last  resort  and  never  as  the  first  option.  There  may  be  theory  or  analytical 
techniques  available  to  solve  a  problem.  When  there  are  they  should  be  used. 

Simulation  can  be  useful  in  situations  that  are  commonly  encountered  in  environmental  planning 
studies,  however.  Some  of  the  reasons  for  using  simulation  include  the  following: 

1)  Simulation  is  the  only  method  available  because  the  actual  environment  is  too 
difficult  to  observe.  (The  fate  of  a  contaminant  introduced  to  a  new  environment 
may  be  impossible  to  track.) 

2)  It  is  not  possible  to  develop  an  analytical  solution. 

3)  Actual  observation  of  a  system  is  too  expensive.  (Planting  tidal  wetlands  with 
different  numbers  and  densities  of  plants  to  determine  the  optimal  scheme  may  be 
too  expensive  to  be  feasible.) 

4)  There  is  insufficient  time  to  allow  the  system  to  operate  extensively.  (We  cannot 
wait  to  see  the  results  of  a  study  of  long-term  trends  in  the  population  of  a  certain 
species.) 

5)  Actual  operation  and  observation  of  a  system  is  too  disruptive.  (If  we  would  like 
to  know  the  impacts  of  a  hiatus  from  maintenance  dredging  on  submerged  aquatic 
vegetation  (SAV)  it  is  not  feasible  to  actually  cease  dredging.) 

6)  Simulation  reveals  new  facts  about  the  problem.  (Building  a  model  requires  careful 
and  logical  thought  about  the  process  under  investigation.) 

7)  Simulation  can  be  an  effective  training  tool.  (It  allows  controlled  experimentation 
with  a  process.) 

8)  Simulation  allows  testing  of  a  greater  range  of  situations  than  have 
been  observed. 


If  you  use  simulation  in  place  of  another  technique  there  are  trade-offs.  Analysts  should  bear  in 
mind  some  of  the  disadvantages  of  simulations.  A  simulation  is  not  an  optimization  process.  It  is  not  precise 
because  you  do  not  get  an  “answer”.  What  you  get  from  a  simulation  is  a  set  (or  distribution)  of  a  system’s 
responses  to  different  operating  conditions. 

Suppose,  for  example,  we  have  a  simulation  model  that  estimates  project  benefits  and  project  costs 
in  order  to  estimate  the  project’s  net  benefits.  Though  we  might  examine  10,000  different  sets  of  operating 
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conditions  and  estimate  the  resulting  costs  10,000  times,  the  resulting  benefits  10,000  times,  and  the 
estimated  net  benefits  10,000  times  we  will  not  know  what  the  actual  net  benefits  are.  We  might  know  the 
average  net  benefits  for  the  10,000  iterations  and  we  might  know  the  minimum  and  maximum  observed 
values,  but  we  will  not  have  a  precise  and  accurate  answer. 

Simulation  models  can  take  a  long  time  to  build  and  they  can  be  very  expensive.  Not  all  situations 
can  be  evaluated  using  simulation.  There  must  be  uncertainty  and  the  uncertainty  must  be  the  kind  that  is 
amenable  to  probabilistic  modeling.  The  method  for  treating  the  uncertainty  depends  in  part  on  the  type  of 
quantity  that  is  uncertain  and  the  source  of  the  uncertainty  as  indicated  in  Chapter  Three.  For  example, 
uncertainty  that  results  from  management’s  inability  to  resolve  questions  is  not  a  proper  subject  for 
simulation  in  many  cases.  Such  instances  are  more  properly  handled  with  sensitivity  analysis. 

Simulations  vary  greatly  in  complexity  but  all  generally  consist  of  the  following  steps: 

1)  Define  the  problem  or  system  to  be  simulated. 

2)  Formulate  the  model  to  be  used. 

3)  Identify  and  collect  the  data  needed  to  test  the  model. 

4)  Test  the  model  and  compare  its  behavior  with  the  behavior  of  the  actual  problem 
environment. 

5)  Run  the  simulation. 

6)  Analyze  the  simulation  results  and  make  changes  to  the  situation/scenario  under 
investigation. 

Flowcharts  are  commonly  used  in  simulations.  They  are  graphic  aids  (like  decision  trees  discussed 
above)  that  help  one  think  logically  about  a  process  and  to  keep  track  of  it.  Process  generators  are  usually 
needed  to  generate  the  data  needed  for  the  simulation.  The  Monte  Carlo  process  is  one  of  the  most 
commonly  used  process  generators. 

The  Monte  Carlo  process  is  a  procedure  that  generates  values  of  a  random  variable  based  on  one  or 
more  specific  probability  distributions,  such  as  those  described  above.  The  Monte  Carlo  process  is  not  a 
simulation  method  or  model  per  se,  although  it  has  become  almost  synonymous  with  stochastic  simulation. 
The  Monte  Carlo  process  begins  with  the  generation  of  a  random  (or  pseudo-random)  number  from  a 
uniform  distribution.  This  number  is  then  transformed  into  a  value  from  the  specific  probability  distribution 
used  by  the  model  builder. 

In  essence,  the  process  works  as  if  you  specify  a  type  of  probability  distribution  and  its  parameters 
and  a  number  is  randomly  selected  from  that  distribution.  Numbers  that  are  common  in  the  specified 
distribution  are  selected  more  often  than  the  numbers  that  are  less  common  in  the  parent  distribution.  If  you 
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select  enough  numbers  from  the  parent  distribution,  you  would  recreate  the  original  distribution  with  your 
sampled  values.  The  random  numbers  generated  by  a  Monte  Carlo  process  can  represent  a  sample  of 
potential  results  in  the  process  under  consideration.  By  contrast,  Latin  Hypercube^®  is  an  alternative  process 
for  generating  random  numbers. 

SPECIALTY  MODELS 

Risk-based  models  have  been  developed  for  a  wide  variety  of  applications.  The  insurance  industry 
has  its  own  set  of  models,  the  chemical  industry  has  developed  numerous  models,  as  has  the  EPA.  The 
Corps  of  Engineers  is  not  without  its  own  specialty  models  for  expected  annual  damage  estimation  and  major 
rehabilitation  of  locks  and  dams. 

Special  models  have  been  developed  for  use  by  the  Corps  in  many  of  their  areas  of  endeavor.  The 
Hydrologic  Engineering  Center  has  developed  numerous  hydrologic  and  hydraulic  models  for  use  by  Corps 
personnel.  HEP,  developed  by  the  U.S.  Fish  and  Wildlife  Service,  is  but  one  of  many  special  models 
developed  for  use  in  the  evaluation  of  habitat.  It  may  well  prove  useful  to  explore  ways  in  which  risk  and 
uncertainty  tools  and  methods  can  be  incorporated  into  a  redesign  of  existing  tools  that  are  serving  their  users 
well.  This  could  include  HSI  models  as  the  example  in  Chapter  Six  illustrates. 

STATISTICS 

Statistics  are  used  to  make  sense  of  data.  Specifically,  they  are  used  to  obtain  information  from  data. 
There  are  two  main  bodies  of  statistics  that  Corps  risk  analysts  need  to  master:  descriptive  and  inferential 
statistics.  Descriptive  statistics  are  used  to  organize,  describe,  summarize,  present  and  understand  data.  This 
includes  measures  of  central  tendency  (means,  medians,  modes),  measures  of  dispersion  (range,  percentiles, 
variation,  skewness,  etc.),  and  tabular  and  graphical  techniques  (contingency  tables/  cross  tabulations,  stem- 
and-leaf  displays,  frequency  distributions,  histograms,  pie  and  bar  charts,  line  graphs,  and  so  on)  used  to 
present  the  information  embodied  in  the  data. 

Frequently,  we  do  not  have  or  cannot  get  information  about  the  entire  population  in  which  we  are 
interested.  For  example,  we  will  never  know  the  true  flow  regime.  At  best  we  will  have  a  sample  of  several 
years  of  data.  It  may  be  impractical  to  measure  the  true  percentage  of  each  type  of  ground  cover  found  in 
the  study  area  for  an  HSI  model.  Instead,  we  may  inventory  a  few  sample  sites  and  use  that  information  to 
make  generalizations  about  the  entire  study  area.  Inferential  statistics  use  information  from  a  sample  to 
generalize  to  the  population  from  which  it  was  drawn. 

Anytime  a  sample  is  used  to  make  some  generalizations  about  the  population  there  is  risk  involved. 
We  are  using  available  and  incomplete  information  to  make  judgments  about  the  population.  There  is  always 


In  a  Latin  Hypercube  process  the  cumulative  probability  distribution  of  an  input  variable  is  stratified  into  n  equal  intervals, 
where  n  is  the  number  of  iterations  in  the  simulation,  along  its  probability  axis.  A  sample  is  drawn  from  each  interval.  Sampling  is  “forced”  to 
represent  values  in  each  interval,  thus  recreating  the  input’s  probability  distribution.  Latin  Hypercube  is  generally  regarded  as  a  more  efficient 
sampling  technique. 
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a  chance  the  sample  is  unrepresentative  of  the  population  from  which  it  came.  An  important  part  of  any  risk 
analysis  is  properly  quantifying  the  risks  associated  with  sampling.  This  is,  in  part,  the  purpose  of  inferential 
statistics. 

Inferential  statistics  involves  the  use  of  sample  statistic  distributions  and  the  calculation  of  interval 
estimates  that  can  be  used  in  sensitivity  analyses  or  Monte  Carlo  simulations.  Interval  estimates  can  be 
calculated  for  various  confidence  levels.  These  same  interval  estimates  can  be  used  in  hypothesis  testing. 
Hypothesis  testing  has  many  applications,  one  of  which  is  testing  the  goodness-of-fit  of  a  data  set  to  a 
probability  distribution. 

It  will  often  be  useful  for  analysts  to  understand  and  be  able  to  estimate  correlations  between  and 
among  variables  that  may  interrelate  in  models  or  calculations  used  in  the  planning  process.  The  need  for 
nonparametric  statistical  techniques  would  be  more  likely  to  arise  in  environmental  planning  than  in  many 
other  professional  settings.  Although  these  techniques  may  not  be  the  analyst’s  highest  priority,  neither 
should  they  be  considered  exotic  techniques. 

Many  of  us  sat  in  our  statistics  classes  and  wondered,  “When  am  I  ever  going  to  need  this  again?” 
One  answer  is,  “When  you  begin  to  incorporate  risk  analysis  into  your  planning  in  order  to  improve  the 
decision-  making  process.”  A  good  command  of  the  basics  of  inferential  statistics  is  essential  for  every  risk 
analyst. 


SUMMARY  AND  LOOK  FORWARD 

The  time  may  come  when  risk  and  uncertainty  protocols  for  environmental  restoration  studies  are 
described  in  detail  in  Corps’  guidance  and  the  models  and  tools  needed  to  conduct  these  analyses  are  well 
known  to  all.  That  day  has  not  yet  arrived  and  so  this  chapter  has  presented  a  list  of  selected  tools  and 
concepts  used  by  many  risk  analysts.  The  chapter  began  with  some  basic  concepts  of  which  environmental 
planning  risk  analysts  will  need  to  be  aware.  These  tend  to  deal  with  information/communication  issues. 
The  latter  half  of  the  chapter  addressed  more  specific  types  of  knowledge  or  tools  that  are  generally 
encountered  by  risk  analysts. 

The  purpose  of  this  chapter  is  not  to  say  these  are  the  techniques  and  here’s  how  you  use  them. 
Rather,  its  intent  is  to  say  here  are  some  techniques  you  might  want  to  learn  more  about.  The  reasons  for 
learning  more  about  them  begin  to  become  apparent  in  the  next  two  chapters.  Chapter  Five  presents  some 
general  sources  of  risk  and  uncertainty  in  environmental  planning.  This  discussion  tends  to  use  the  broad 
concepts  found  in  this  chapter.  Chapter  Six,  by  contrast,  discusses  more  specific  examples  of  risk  and 
uncertainty  encountered  in  environmental  planning.  These  examples  make  more  use  of  the  specific  tools 
presented  here. 
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CHAPTER  FIVE:  GENERAL  SOURCES  OF  RISK  AND 
UNCER  TAINTY  IN  ENVIRONMENTAL 
RESTORA  TION  PLANNING 

INTRODUCTION 

.  .  it  is  impossible  to  predict  with  certainty  how  animal  populations  will  respond  to 
restored  or  improved  hydrological  conditions.”  Central  and  South  Florida  Project 
Reconnaissance  Report  Comprehensive  Review  Study,  November  1994,  p.  108,  U.S.  Army 
Corps  of  Engineers,  Jacksonville  District. 

The  above  quotation  sums  up  the  frustration  of  the  environmental  planner’s  job  quite  well.  Their 
task  is  a  difficult  one.  Many  things  are  uncertain  in  their  studies.  The  best  way  to  proceed  is  to  acknowledge 
those  uncertainties,  identify  the  most  important  among  them,  and  address  them  in  a  rational  fashion,  subject 
to  study  constraints.  This  chapter  identifies  sources  of  risk  and  uncertainty  that  could  be  generic  to  virtually 
any  water  resources  planning  project.  They  are  also  the  broad  concepts  of  risk  and  uncertainty  encountered 
in  environmental  planning. 

An  earlier  research  report^'  reviewed  a  number  of  Corps  environmental  project  reports.  Very  little 
evidence  of  risk  and  uncertainty  analysis  was  found  in  them.  A  number  of  environmental  project  reports  were 
reviewed  for  this  effort  with  a  similar  result.  There  is  considerable  evidence  of  uncertainty  in  the  project 
reports.  Frequently,  the  limits  of  the  analysts’  knowledge  are  straightforwardly  acknowledged.  For  example, 
comments  like  “.  .  .  insufficient  information  is  available  to  make  definitive  statements  about  the  habitat 
requirements  of  hickory  shad. . .”  are  not  uncommon.  This  acknowledgment  of  the  existence  of  uncertainty 
is  an  important  first  step  in  doing  something  about  it. 

Despite  some  acknowledgment  of  uncertainty  it  is  still  more  common  to  see  very  precise  values 
presented  as  if  they  are  absolute  certainties.  For  example,  in  one  study  17,28 1  habitat  units  were  reportedly 
affected  in  the  past.  Though  an  incredibly  precise  number,  it  is  pure  folly  to  believe  that  number  is  anything 
more  than  an  approximation  of  conditions  that  used  to  exist.  There  is  still  a  great  deal  to  be  done  in 
acknowledging  the  limits  of  our  knowledge. 

This  report’s  review  of  project  documents  made  it  clear  that  there  were  different  categories  of 
sources  of  uncertainty  and  different  places  where  risk  and  uncertainty  analyses  could  be  introduced. 
Ignorance  of  the  planning  process  is  one  of  the  greatest  sources  of  uncertainty  evident  in  many  reports. 
Many  reports  did  not  have  clearly  identified  problems  and  relatively  few  had  any  planning  objectives.  Absent 
these  fundamentals,  it  was  often  difficult  to  know  what  objectives  the  plans  were  formulated  to  meet  and 
what  problems  they  might  solve.  These  shortcomings  are  not  in  any  way  unique  to  environmental  planning. 


21 

Compilation  and  Review  of  Completed  Restoration  and  Mitigation  Studies  in  Developing  an  Evaluation  Framework  for 
Environmental  Resources,  Volumes  I  and  11,  IWR  Reports  #95-R-4  and  5. 
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There  are,  however,  uncertainties  that  are  unique  to  the  environmental  restoration  process.  One  of 
these  is  the  entire  habitat  evaluation  process.  Another  is  the,  “If  we  build  it,  will  they  come?”  question,  i.e., 
will  projects  actually  result  in  environmental  improvements?  With  issues  running  from  the  general  to  the 
very  specific  it  is  necessary  to  provide  some  order  to  the  manner  in  which  these  issues  are  to  be  discussed. 
Some  general  sources  of  risk  and  uncertainty  are  presented  in  this  chapter.  They  are  presented  here  because 
they  may  be  expected  to  occur  with  greater  regularity,  and  perhaps  with  more  impact,  in  environmental 
restoration  studies. 

To  organize  the  presentation,  the  P&G-mandated  six-step  planning  process  used  by  the  Corps  in  its 
traditional  planning  work  provides  the  organizational  structure  for  the  chapter.  Those  steps  are  presented  in 
Figure  1.  The  chapter  is  divided  into  ten  sections:  this  introduction,  some  general  observations,  one  section 
for  each  planning  step,  and  two  summary  sections.  Discussion  related  to  specific  environmental  restoration 
issues  are  covered  in  Chapter  Six. 

FOUR  GENERAL  OBSERVATIONS 

Before  some  broad  concepts  of  risk  and  uncertainty  are  discussed,  a  few  observations,  too  general 
to  fit  elsewhere,  are  introduced  here.  First,  risk  analysis  is  not  intended  to  increase  the  work  requirements 
for  anyone.  Any  risk  analysis  must  be  purposeful,  i.e.,  it  must  contribute  to  improved  decision-making.  In 
order  to  be  purposeful,  we  need  to  examine  and  understand  what  is  already  being  done  by  environmental 
planners.  That  process  only  begins  with  this  report. 

The  second  observation  follows  from  the  first.  There  is  a  need  to  identify  critical  variables  and  key 
relationships  in  what  planners  are  already  doing  in  environmental  restoration  studies.  This  will  take  more 
time  and  focussed  effort  than  were  possible  in  this  initial  step  in  the  process  of  introducing  risk  and 
uncertainty  analysis  to  environmental  planning. 

A  third  observation  is  not  specifically  related  to  risk  and  uncertainty.  Many  planners  do  not  know 
about  or  understand  the  P&G  planning  process.  When  planners  don’t  know  how  to  go  about  planning  there 
is  a  good  chance  there  will  be  considerable  uncertainty  about  what  they  have  done. 

The  last  general  observation  is  a  simple  one.  Assumptions  must  constantly  be  made  by  planners  in 
order  for  work  to  proceed.  Professional  Judgment,  intuition  and  gut  feelings  are  important  tools. 
Assumptions  should  be  avoided  where  answers  exist.  The  need  for  and  rationale  of  assumptions  that  are 
made  should  always  be  clear,  as  the  assumptions  themselves  need  to  be  clear. 

These  are  “big  picture”  uncertainties  as  is  most  of  what  follows  in  this  chapter.  Much  of  what 
follows  considers  uncertainty  about  the  decision-making  process  itself.  If  the  process  by  which  decisions 
are  made  is  uncertain  or  poorly  understood,  the  decisions  that  stem  from  it  are  questionable.  Fundamental 
uncertainties  such  as,  “Are  we  addressing  the  right  problems?”  or  “Do  we  have  the  best  solution?”  are 
indeed  broad  in  concept.  But  they  are  far  from  trivial. 
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STEP  ONE:  PROBLEM  IDENTIFICATION 

There  should  be  no  uncertainty  about  the  purpose  and  objectives  of  an  environmental  restoration 
study.  The  first  step  in  the  planing  process  is  the  most  critical  one.  In  this  step,  planners  identify  the 
problems  that  have  motivated  the  study.  Problems  may  be  identified  by  the  public  or  by  technical  experts. 
In  addition  to  problems,  planners  should  identify  opportunities  for  improving  conditions  now  and  into  the 
future.  Building  on  the  identification  of  problems  and  opportunities,  the  planning  team  develops  specific 
objectives  they  want  to  achieve  in  the  planning  process.  At  the  same  time,  they  identify  constraints,  i.e., 
those  things  they  want  their  plans  to  avoid  doing. 

The  planning  objectives  and  constraints  effectively  become  the  mission  statement  for  the 
Federal/non-Federal  planning  partnership.  They  are  a  concise  and  unique  statement  of  the  reasons  these  two 
parties  have  come  together  in  this  study  effort.  Plans  are  formulated,  evaluated,  compared  and  selected  based 
on  their  contributions  to  the  planning  objectives. 

A  review  of  15  environmental  restoration  project  studies  in  preparation  of  this  report  revealed  an 
uneven  record  in  terms  of  their  faithfulness  to  this  planning  process.  Some  studies  used  a  study  team.  Other 
studies  were  done  almost  entirely  by  an  individual.  Problem  identification  was  the  part  of  this  first  planning 
step  that  was  generally  handled  best.  Although  there  was  rarely  a  clear  and  concise  statement  of  the 
problems  with  the  kind  of  directness  found  in  statements  like,  “The  problem  is. . one  usually  understood 
the  problem  after  reading  the  report. 

Cause  and  effect  linkages  were  frequently  missing  from  problem  discussions.  A  somewhat  typical 
problem  identification  follows: 

“There  is  a  deficiency  of  at  least  15  acres  of  barrier  island  habitat  at  or  above  the  10-year 

flood  elevation,  with  stabilizing  vegetation.” 

Statements  like  these  do  a  good  job  of  specifically  identifying  problems  -  lack  of  a  specific  habitat  type. 
They  are  less  effective  in  explaining  why  15  acres  is  singled  out  and  they  do  not  clearly  establish  the  cause 
of  the  problem.  Frequently,  these  additional  details  can  be  ferreted  out  from  the  surrounding  text.  That  is 
not  always  the  case.  When  it’s  not,  problems  remain  uncertain.  When  problems  are  uncertain  so  are  the 
solutions. 

The  manner  in  which  problems  are  discussed  and  described,  largely  absent  clear  and  concise 
descriptive  problem  statements,  can  contribute  to  uncertainty  over  what  the  problems  are  perceived  to  be. 
We  might  all  agree  there  is  a  lack  of  a  certain  habitat  type,  but  we  might  each  have  a  different  idea  about  why 
that  is  so.  This  potential  uncertainty  can  be  limited  by  a  careful  problem  statement  that  includes  a  cause  and 
effect  linkage. 

This  is  not  a  trivial  matter.  It  may  be  easy  to  note  a  decrease  in  duck  habitat  that  accompanies  a 
decline  in  migratory  duck  populations.  The  temptation  to  link  the  two  is  strong.  However,  understanding 
the  problem  is  essential.  If  the  decline  in  ducks  is  due  to  overhunting  over  a  multi-state  region,  then  no 
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amount  of  habitat  restoration  will  solve  the  problem.  In  order  for  plans  to  be  effective,  studies  must  begin 
with  clear  and  definitive  problem  statements,  devoid  of  ambiguity  or  other  uncertainty. 

The  reports  reviewed  tended  to  do  less  well  with  other  important  aspects  of  this  first  planning  step. 
Solving  problems  is  clearly  one  way  to  make  things  better.  Seizing  opportunities  is  another.  Opportunities 
are  rarely  addressed.  And  yet,  altering  projects  to  seize  opportunities  seems  very  much  in  concert  with  the 
spirit  of  environmental  restoration.  As  a  result,  there  can  be  uncertainty  about  the  planners’  responsiveness 
to  potential  opportunities. 

Another  common  shortcoming  is  the  manner  in  which  planning  objectives  and  constraints  are 
handled.  Too  often,  there  are  no  clearly  stated  planning  objectives.  Too  many  reports  lean  on  discussions 
of  the  NED  objective  which,  though  a  Federal  objective,  is  not  a  planning  objective.  In  other  cases  the 
objectives  remain  vague,  for  example,  “. . .  to  preserve  and  enhance  the  existing  aquatic  plant  beds  for  fish 
and  wildlife  habitat,”  to  be  truly  useful  in  plan  formulation.  When  this  happens  there  is  substantial 
uncertainty  about  the  objectives  of  the  entire  planning  process. 

At  the  other  extreme,  objectives  sometimes  include  plan  formulation  and  selection.  For  example, 
an  objective  that  said,  “Re-establish  15  acres  of  stabilized  islands  from  river  miles  300  to  300.8”,  is  too 
specific.  Objectives  should  never  include  solutions. 

One  of  the  most  glaring  offenses  of  the  planning  reports  is  that  once  planning  objectives  are 
identified  there  is  no  apparent  further  use  of  them  in  the  planning  process  in  the  majority  of  the  reports 
reviewed.  This  is  one  of  the  more  troubling  indications  of  a  potential  general  ignorance  of  the  Corps’ 
planning  process.  The  work  done  in  this  first  step  is  essential  to  the  success  of  the  planning  process. 
Identifying  problems  and  opportunities  and  specifying  objectives  and  constraints  are  essential  steps  in 
limiting  the  uncertainty  of  purpose  that  can  be  encountered  in  a  planning  study. 

There  are,  on  occasions,  studies  in  which  it’s  never  really  clear  what  is  being  accomplished,  despite 
the  copious  details  on  project  design  and  increased  habitat  units.  This  may  be  due  to  ignorance  of  the 
planning  process.  Some  planning  steps  may  not  have  been  taken  or  were  taken  incorrectly.  In  other  cases, 
the  report  writers  have  simply  failed  to  tell  their  story  effectively.  In  either  case,  it  is  in  everyone’s  interests 
that  planners  eliminate  all  uncertainty  about  the  fundamental  purposes  of  the  study  effort.  Knowledge  of 
the  planning  process  and  a  clear  “story  telling”  style  of  documentation  are  the  best  ways  to  eliminate  or 
reduce  this  potential  uncertainty  of  purpose. 

STEP  TWO:  INVENTORY  AND  FORECAST 

In  the  second  step  of  the  planning  process,  planners  identify  the  data  they’ll  need  to  solve  problems, 
seize  opportunities  and  formulate  plans  to  meet  their  planning  objectives.  At  this  step,  information  is 
gathered  in  sufficient  detail  to  adequately  define  the  existing  conditions  and  to  define  a  most  likely  future 
condition  without  a  project. 
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Uncertainty  is  removed  by  information.  So,  it  should  be  evident  that  this  information  gathering  step 
is  an  essential  one  in  any  attempt  to  limit  uncertainty.  It  should  be  equally  evident  that  any  attempt  to 
describe  a  future  condition  is  fundamentally  an  exercise  in  uncertainty. 

At  times,  environmental  restoration  studies  may  rely  too  much  on  “cut  and  paste”  data.  This  may 
seem  a  contradiction  when  Section  1 135  studies  are  expected  to  make  use  of  existing  data  sources.  The 
problem  is  not  so  much  using  existing  data  as  much  as  it  is  using  data  because  it  exists.  In  some  cases,  the 
recitation  of  existing  condition  data  have  no  material  bearing  on  the  problems,  opportunities,  objectives, 
plans  or  their  effects.  When  that’s  the  case,  the  data  are  extraneous  and  should  not  be  used. 

If  these  data  are  used  in  lieu  of  the  data  required  to  enlighten  the  decision  process,  uncertainty  will 
persist.  There  are  countless  potential  sources  of  uncertainty.  They  include  professional  judgments, 
professional  disagreements,  lack  of  information,  inaccessibility  of  private  property,  interpolations  and 
extrapolations  of  existing  information,  failure  to  recognize  limiting  factors,  and  on  and  on.  For  example, 
measurements  of  habitat  variables  critical  to  the  estimation  of  habitat  unit  changes  attributable  to  a  plan 
frequently  appear  in  precise  point  estimates  that  cover  the  data  in  a  mantle  of  false  precision  and  certainty. 
This  issue  is  taken  up  in  Chapter  Six. 

To  a  very  great  extent,  efforts  to  quantify  past  conditions  are,  at  best,  guesswork.  Rarely  will 
planners  have  sufficient  information  to  describe  past  environmental  conditions  with  certainty.  What  would 
the  current  state  of  the  environment  have  been  in  the  absence  of  the  Corps  project?  Is  that  what  is  motivating 
the  restoration  study?  Would  it  have  been  different  without  the  project?  If  so,  would  it  have  been  better  or 
worse?  When  we  consider  the  cumulative  impact  of  all  the  stresses  on  an  ecosystem,  it  is  not  always  evident 
that  such  a  discussion  has  any  real  meaning  for  a  complex,  dynamic  ecosystem.  Thus,  it  may  be  better  to 
acknowledge  the  uncertainty  inherent  in  this  backward  looking  analysis  and  to  concentrate  resources  more 
on  solving  problems  and  seizing  opportunities. 

One  of  the  more  common  situations  in  which  uncertain  values  are  left  to  stand  unchallenged  is  when 
planners  must  rely  on  the  judgments  of  professionals  to  describe  past,  present,  and  future  conditions.  These 
judgments  are  used  in  the  absence  of  better  information  but  they  are  rarely  varied  or  tested.  Planners 
infrequently  say  why  this  judgment  is  best  and  they  never  say  why  it  is  the  only  possibility.  The  accuracy 
of  subjective  data  and  judgments  is  always  improved  by  providing  an  interval  estimate.  We’re  always  more 
likely  to  be  right  when  we  can  say  an  uncertain  thing  is  between  “this  and  that”  rather  than  by  stating 
precisely  what  it  will  be. 

A  complicating  factor  in  identifying  existing  and  future  conditions  for  environmental  restoration 
studies  is  that  a  static  condition  is  inadequate  for  describing  a  dynamic  system.  What  does  it  mean  to 
describe  the  natural  conditions  of  a  dynamic  ecosystem  when  conditions  are  constantly  changing  in  nature? 
The  point  of  these  and  other  such  questions  is  not  to  bog  analysis  down  in  esoteric  questions  as  much  as  it 
is  to  get  planners  who  work  in  the  field  of  environmental  restoration  to  begin  to  develop  some  introspection 
and  insight  into  what  is  uncertain  in  the  work  they  do  so  that  they  can  identify  those  uncertainties  that  are 
most  important;  then  find  ways  to  address  them. 
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STEP  THREE:  FORMULATE  ALTERNATIVE  PLANS 

How  can  we  have  the  best  plan  unless  it’s  chosen  from  the  set  of  best  alternatives?  If  there  is  not 
a  clear  set  of  planning  objectives,  what  guides  the  plan  formulation  process?  How  can  there  be  a  certain  link 
between  problems  and  solutions  if  we  do  not  have,  or  have  but  don’t  use,  planning  objectives?  From  where 
do  plans  come?  How  do  you  know  you  have  a  good  set  of  alternatives?  Having  the  best  plan  is  a  “big 
picture”  uncertainty,  but  it  is  most  certainly  not  a  trivial  issue. 

The  greatest  source  of  uncertainty  in  this  step  is,  once  again,  ignorance  of  the  planning  process.  If 
there  is  not  a  principled  and  structured  approach  to  solving  problems  and  seizing  opportunities  there  is  no 
way  to  know  if  an  implementable  project  is  really  the  best  one  or  even  a  good  one.  Most  project  reports 
offered  little  evidence  of  a  thorough  formulation  process.  The  choice  of  a  plan  is  the  most  important 
decision  made  in  the  planning  process.  If  risk  and  uncertainty  analysis  is  intended  to  improve  the  quality 
of  the  decisions  as  well  as  the  decision  process,  it  is  a  tautological  fact  that  the  best  choice  can  only  come 
from  the  best  set  of  alternatives.  Planners  can  reduce  the  uncertainty  about  whether  they  have  identified  the 
best  plan  by  eliminating  the  uncertainty  attending  their  planning  objectives  and  by  providing  as  broad  a  range 
of  alternatives  as  possible. 


STEP  FOUR:  EVALUATE  PLANS 

The  evaluation  of  plans  requires  well  defined  planning  objectives  and  well  formulated  plans.  This 
step  is  subject  to  many  of  the  generic  uncertainties  already  mentioned.  A  with  project  future  condition  must 
be  identified  for  each  alternative  plan.  This  will  require  data,  judgment  and  guesswork  because  it,  like 
identification  of  the  without  project  future  condition,  is  a  fundamentally  uncertain  exercise. 

The  decision  process  becomes  even  more  uncertain  in  this  step,  however,  because  it  requires 
assumptions  about  project  performance.  The  performance  of  environmental  restoration  projects  may  be 
subject  to  more  uncertainty  than  most  projects,  a  topic  taken  up  again  in  Chapter  Six. 

Any  project  is  subject  to  uncertainties  about  costs  and  design.  A  common  problem  encountered  in 
measuring  plan  effects  is  the  tendency  to  substitute  precision  for  accuracy.  It  may  be  comforting  to  see  the 
cost  of  earth  moving  precisely  estimated  to  be  $2.45  per  cubic  yard  (CY).  The  accuracy  of  this  cost  is 
another  question.  Do  we  really  expect  the  cost  per  CY  to  be  $2.45?  How  often  are  costs  as  accurate  as  they 
are  precise? 

Generally,  estimators  work  with  costs  from  a  number  of  similar  jobs.  These  costs  tend  to  cover  a 
range  of  values.  The  estimator’s  expertise  comes  into  play  in  how  he  develops  a  point  estimate  from  all  the 
information  gathered.  The  quality  of  the  decision  might  actually  be  improved,  however,  if  costs  were 
estimated  as  a  range.  Then,  decision-makers  could  see  the  range  that  costs  could  eventually  span.  Such 
information  could  also  be  used  by  analysts  in  combination  with  other  information  to  provide  a  better  estimate 
of  the  possible  true  range  in  costs  per  habitat  unit  produced  by  the  project. 
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Environmental  restoration  projects  may  be  more  uncertain  than  other  studies  because  there  has  been 
relatively  less  experience  in  building  and  maintaining  them.  Many  things  being  done  may  be  done  for  the 
first  time.  Even  when  planners  are  experienced  it  is  not  uncommon  to  run  into  substantial  design  and  cost 
uncertainties  because  of  a  lack  of  foundation  or  hydrologic  and  hydraulic  data,  and  so  on.  Planners  might 
be  wise  to  expect  these  common  sources  of  uncertainty  to  be  exacerbated  in  unique  environmental  restoration 
studies. 


STEP  FIVE:  COMPARE  PLANS 

Once  plan  effects  have  been  evaluated  it  is  necessary  to  compare  the  effects  of  one  plan  to  the  effects 
of  another  plan  as  well  as  to  the  option  of  taking  no  action.  A  comparison  of  plan  effects  requires  some 
criteria  upon  which  the  comparison  will  be  based.  Once  again,  those  who  are  ignorant  of  the  planning 
process  may  be  laboring  in  the  shadows  of  uncertainty.  If  the  criteria  upon  which  the  comparison  is  based 
are  not  known  to  decision-makers  there  is  the  potential  for  considerable  misunderstanding  and  error  in  the 
decision  process. 

The  plans’  relative  contributions  to  the  planning  objectives  provide  the  basis  for  the  most  objective 
comparison  of  plans.  What  is  not  known  are  the  weights  that  stakeholders  will  place  on  the  different  plans’ 
effects  and  their  contributions  to  the  various  planning  objectives.  It  may  be  easy  to  say  that  more  of  one 
thing  beats  less  of  that  thing  but  it  becomes  very  difficult  to  trade-off  differences  in  the  mix  of  contributions 
plans  make  to  planning  objectives.  Thus,  clarity  and  certainty  of  purpose  in  the  study  as  well  as  in  the 
explanation  and  display  of  results  are  essential  to  the  reduction  of  common  sources-  of  uncertainty  in  many 
planning  studies. 

There  is  an  additional  source  of  uncertainty  in  environmental  restoration  planning.  Not  all  of  the 
benefits  of  these  projects  can  be  measured  in  dollar  terms.  This  lack  of  a  common  metric  means  there  is  less 
reliance  on  the  national  economic  development  criterion  of  net  benefits  and  more  reliance  on  other  criteria 
that  may  be  less  familiar  to  planners  and  stakeholders. 

STEP  SIX:  SELECT  A  PLAN 

In  the  final  step  of  the  planning  process  a  course  of  action  is  chosen.  This  usually  means  the 
selection  of  a  plan,  although  no  action  is  always  an  option.  At  this  step  in  the  planning  process,  all  the 
uncertainties  encountered  in  the  study  become  cumulatively  and  hopelessly  hidden  from  the  view  of 
unsuspecting  decision-makers  unless  there  has  been  a  systematic  attempt  to  deal  with  risk  and  uncertainty 
throughout  the  process. 

If  significant  issues  have  been  obscured  from  the  view  of  decision-makers  there  is  a  real  possibility 
that  an  error  will  be  made  in  plan  selection.  Bad  investments  may  be  made.  Good  investments  may  be 
overlooked.  The  sole  purpose  of  introducing  risk  and  uncertainty  analysis  to  environmental  restoration 
planning  is  to  improve  decisions.  That  happens  in  this  step,  but  only  if  it  has  taken  place  throughout  the 
planning  process. 
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None  of  the  sources  of  risk  and  uncertainty  discussed  in  this  section  are  unique  to  environmental 
restoration  projects.  Inasmuch  as  the  newness  of  environmental  restoration  challenges  planners  at  many 
levels,  it  would  seem  all  the  more  important  to  engage  in  a  structured,  rational  planning  process  as  devoid 
of  uncertainty  as  possible. 

WHAT  IS  TO  BE  DONE  ABOUT  GENERAL  SOURCES  OF 
RISK  AND  UNCERTAINTY? 

This  chapter  is  not  an  indictment  of  the  state  of  planning.  It  is  instead  an  endorsement  of  the  Corps’ 
planning  process.  Virtually  all  of  the  sources  of  uncertainty  identified  in  the  preceding  sections  can  be 
eliminated  through  the  knowledge  and  use  of  the  six-step  planning  process  promulgated  by  the  Principles 
and  Guidelines. 

A  structured,  rational  framework  for  approaching  environmental  restoration  problems  can  eliminate 
confusion  about  the  purpose  and  the  process  of  planning  for  planners  and  stakeholders  alike.  This  would 
free  planners  to  coneentrate  on  those  specific  elements  of  a  study  in  which  risk  and  uncertainty  analysis  can 
be  systematically  applied  in  order  to  improve  the  quality  of  decision-making. 

SUMMARY  AND  LOOK  FORWARD 

Just  as  there  are  broad  concepts  that  can  be  used  in  risk  and  uncertainty  analysis  there  are  “big 
picture”  issues  of  uncertainty  encountered  in  environmental  planning  studies.  These  issues  are  not  unique 
to  environmental  planning.  In  fact,  they  are  more  indigenous  to  the  planning  process  than  they  are  to  the 
environment.  Uncertainties  are  encountered  in  each  step  of  the  planning  process  and  they  can  become 
cumulative  if  not  addressed,  virtually  assuring  that  decision-makers  labor  in  ignorance  of  potentially 
important  issues.  Nonetheless,  the  uniqueness  and  complexity  of  environmental  plans  suggests  that  the 
uncertainties  inherent  in  the  planning  process  may  become  particularly  acute  in  environmental  planning. 

In  addition  to  the  big  picture  uncertainties,  there  are  many  uncertainties  specific  to  environmental 
planning.  Some  of  these  are  discussed  in  the  chapter  that  follows.  An  example  introducing  risk-based 
analysis  to  the  estimation  of  habitat  unit  changes  is  also  offered  to  demonstrate  the  feasibility  of  some  of  the 
methods  presented  in  the  last  chapter  to  addressing  some  of  the  problems  of  the  next. 
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CHAPTER  SIX:  SELECTED  SPECIFIC  SOURCES  OF  RISK 
AND  UNCERTAINTY  IN  ENVIRONMENTAL 
RESTORA  TION  PLANNING 

INTRODUCTION 

This  chapter  considers  potential  sources  of  risk  and  uncertainty  that  are  specific,  if  not  necessarily 
unique,  to  environmental  restoration  projects.  These  sources  of  risk  and  uncertainty  have  been  grouped  for 
convenience  into  four  topics  for  discussion.  These  are  project  performance,  models,  extrapolation,  and,  the 
habitat  evaluation  procedures  (HEP)  process.  The  HEP  process  is  the  primary  focus  of  this  chapter  because 
it  is  one  of  the  primary  tools  of  environmental  restoration  analyses  and,  as  such,  is  a  logical  point  at  which 
to  begin  looking  for  ways  to  introduce  risk  and  uncertainty  analysis. 

PROJECT  PERFORMANCE 

Project  performance  involves  some  fundamentally  uncertain  issues.  They  range  from  questions 
about  what  environmental  restoration  projects  produce  to  whether  the  management  measures  will  result  in 
what  is  wanted.  This  is  the  most  basic  environmental  restoration  uncertainty.  What  are  environmental 
projects  supposed  to  do  and  do  they  work?  There  was  a  surprising  candor  about  the  relevance  of  these 
questions  encountered  among  Corps  personnel  interviewed  during  the  preparation  of  this  report. 

PROJECT  OUTPUTS 

Many  environmental  restoration  projects  increase  habitat  units.  But,  what  does  that  mean?  The 
actual  outputs  of  environmental  projects  were  not  often  identified  in  the  reports  reviewed  for  this  document 
despite  the  fact  that  EC  1 1 05-2-206  Project  Modifications  for  Improvement  of  the  Environment  requires  them 
to  be.  When  they  are  identified,  they  are  frequently  poorly  specified.  For  example,  one  Section  1135 
report’s  sole  description  of  project  outputs  was  “. . .  aquatic  insects  would  benefit. . .”.  Without  judging  the 
worthiness  of  such  an  output,  most  would  agree  that  it  may  have  been  useful  to  identify  the  types  of  insects 
that  would  benefit,  the  extent  of  their  benefit  and  the  significance  of  that  benefit. 

There  is  no  reason  for  project  outputs  to  remain  uncertain  to  either  planners,  decision-makers,  or 
the  readers  of  project  reports.  Although  the  outputs  of  flood  damage  reduction  or  navigation  projects  may 
be  more  or  less  taken  for  granted,  outputs  of  environmental  restoration  projects  need  to  be  specified 
especially  because  they  are  new.  Although  habitat  units  may  be  one  measure  of  project  outputs,  it  should 
be  clear  that  what  a  project  produces  is  far  more  specific  than  simple  HU’s.  They  should  be  identified  as 
completely  as  possible. 

To  do  this  well  requires  good  planning.  If  problems  and  opportunities  are  carefully  identified  in  the 
planning  process,  planning  objectives  can  be  specified.  Achieving  the  planning  objectives  is  the  purpose  of 
any  plan.  Hence,  careful  identification  of  planning  objectives  is  an  essential  step  in  the  specification  of 
project  outputs.  Estimating  habitat  units  is  simply  one  means  of  evaluating  and  comparing  the  effects  of 
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plans  developed  during  the  planning  process.  Outputs  can  be  measured  in  any  number  of  ways,  including 
habitat  units,  population  counts,  biological  integrity,  environmental  values,  energy  flow,  physical  and 
chemical  characteristics  of  water,  and  other  ecosystem  components.  The  Corps’  Evaluation  of 
Environmental  Investments  Research  Program  has  an  ongoing  work  unit,  “Determining  Objectives  and 
Measuring  Outputs”,  that  addresses  some  of  these  concerns. 

Too  often,  the  vagueness  or  uncertainty  surrounding  project  outputs  is  due  to  incomplete  planning. 
It  is  impossible  to  develop  good  plans  unless  the  objectives  of  those  plans  are  clearly  identified.  Project 
outputs  are  contributions  to  the  planning  objectives.  These  should  never  be  uncertain  and  they  should  always 
be  clearly  identified  in  the  study  documentation. 

EFFECTIVENESS  OF  PLANS 

A  number  of  environmentalists  and  planners  interviewed  during  the  preparation  of  this  report 
identified  project  performance  as  a  fundamental  source  of  uncertainty.  One  planner,  paraphrased  below, 
described  the  uncertainty  rather  succinctly.  “We’re  fiddling  with  landscape  -  the  spacing  of  plants,  height 
of  fill,  number  of  duck  boxes  -  and  we’re  using  our  hydraulics  and  hydrology  and  water  quality  data  to 
manipulate  water.  Will  this  result  in  what  we  want?  If  we  build  it,  will  they  come?” 

Can  we  manage  salinity  and  water  levels  to  increase  diversity  in  the  aquatic  plant  community  for 
wildlife?  Dozens  of  such  questions  can  be  posed.  In  some  cases  we  can  pinpoint  projects  that  are 
remarkable  successes.  In  other  instances  there  have  been  abject  failures.  In  all  honesty,  planners  may  not 
always  know  how  well,  if  at  all,  their  projects  will  succeed  in  achieving  the  planning  objectives  and 
producing  the  desired  project  outputs. 

The  uncertainty  of  project  performance  can  be  addressed  in  a  number  of  ways.  In  some  cases, 
planners  have  adjacent  areas  or  similar  projects  they  can  use  to  gage  project  performance  and  place  more 
reasonable  bounds  on  the  uncertainty  accompanying  project  performance.  When  such  projects  are  not 
available,  sensitivity  analysis  can  be  used  to  test  project  performance. 

For  example,  once  the  most  likely  future  scenario  has  been  used  to  evaluate  plan  effects,  reductions 
in  critical  outputs  can  be  considered.  One  might  ask,  if  water  quality  improvements  are  only  90  percent  of 
what  we  expect  them  to  be,  would  we  continue  with  the  project?  How  about  if  they  are  80,  70  or  60  percent 
of  what  we  expect?  Simple  consideration  of  alternative  outputs  like  this  can  help  identify  thresholds  for  the 
plan’s  success  or  for  project  support.  Then  the  likelihood  of  those  thresholds  can  be  subjectively  estimated 
to  determine  if  this  presents  a  significant  possibility.  See  the  sidebar  on  the  next  page  for  an  example. 

Sensitivity  analysis  can  be  more  sophisticated.  Critical  variables  in  the  evaluation  and  comparison 
of  plan  effects  can  be  systematically  varied  or  simulations  can  be  used  to  model  potential  outcomes.  An 
example  illustrating  how  estimates  of  habitat  units  can  vary  is  provided  later  in  this  chapter. 
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In  some  instances,  the  ecosystems  are  so  complex  and  the  solutions  so  experimental  that  monitoring 
of  project  performance  after  the  fact  is  the  only  way  to  reasonably  gage  project  performance.  In  these 
instances,  the  uncertainty  of  project  performance  would  best  be  acknowledged  during  the  study  and  used  to 
design  a  satisfactory  monitoring  program.  As 
more  projects  are  constructed  and  monitored  the 
bounds  of  uncertainty  about  project  performance 
will  be  narrowed. 

Some  uncertainty  over  project 
performance  can  be  addressed  through  simple 
sensitivity  analysis.  In  other  cases  more 
sophisticated  and  systematic  analyses  may  be 
helpful.  In  still  other  cases,  monitoring  of 
experimental  projects  may  be  the  most 
appropriate  way  to  handle  uncertain  project 
performance.  In  any  event,  one  or  more  of  these 
techniques  are  within  easy  grasp  of  Corps 
planners. 

EXTRAPOLATION 

Extrapolation  is  extending  judgments  or 
conclusions  into  an  area  that  is  not  known  or 
experienced.  It  is  conjectural  knowledge  and  it’s 
everywhere  in  environmental  restoration  projects. 

Under  the  best  circumstances,  extrapolation  is 
scientific  guesswork.  It  is  often  the  best  we  can  do  given  the  present  limits  to  our  knowledge.  Planners  are 
forced  to  extrapolate  because  the  systems  they  work  with  are  so  complex  and  our  knowledge  of  them  is  so 
incomplete.  Under  the  worst  circumstances,  extrapolation  can  result  from  lack  of  knowledge  or  a  lack  of 
resources  required  to  complete  a  study. 

The  very  notion  of  identifying  a  target  species  and  then  using  the  condition  of  the  habitat  for  this 
species  as  an  indicator  of  the  ecosystem’s  condition  involves  considerable  conjecture.  The  assumed  linear 
relationship  between  an  HSI  and  carrying  capacity,  that  is  built  into  this  approach,  is  also  conjecture. 
Extrapolating  the  results  of  a  project  at  one  location  to  expected  results  of  a  project  at  another  location  is 
redolent  with  uncertainty,  especially  when  there  are  significant  differences  between  the  locations. 

Even  fundamental  cause  and  effect  relationships  are  frequently  extrapolated.  For  example,  a 
decrease  in  habitat  accompanied  by  a  decrease  in  migratory  ducks  would  often  be  considered  prima  facie 
evidence  that  habitat  loss  is  causing  duck  losses.  More  careful  study  might  indicate  duck  losses  are  due  to 
overhunting  and  duck  habitat  is  being  lost  because,  as  the  ducks  in  this  part  of  the  country  have  disappeared 
over  time,  the  opportunities  (and  hence  the  demand)  for  hunting,  have  fallen  off.  As  a  result,  land  that  was 


Simple  Sensitivity 

Suppose  a  project  to  produce  a  freshwater 
marsh  has  a  salinity  goal  of  0-0.5  parts  per  thousand 
(ppt).  If  the  project  results  in  1  ppt  would  it  still  be 
constructed?  How  about  1.5  ppt?  Let’s  further  suppose 
that  such  a  thought  process  leads  the  study  team  to 
conclude  that  salinities  could  reach  as  high  as  4  ppt 
before  the  project  would  be  considered  undesirable, 
based  on  the  plan’s  contributions  to  the  full  set  of 
planning  objectives. 

The  next  step  in  the  sensitivity  analysis  would 
be  to  try  and  gage  the  likelihood  of  the  events  that 
would  have  to  transpire  to  result  in  a  4  ppt  salinity  level. 
If  these  are  Judged  to  be  unlikely,  the  plan  can  proceed 
with  relative  confidence  that  the  project  will  at  least 
achieve  acceptable  salinity  levels.  If  the  likelihood  is 
greater  than  decision-  makers  would  like  then  perhaps 
the  plan  requires  some  fine  tuning  or  a  more  detailed 
risk  analysis. 

k  ■■  . 
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once  preserved  for  hunting  the  ducks  that  populated  it  may  now  be  put  to  alternative  uses.  To  some,  these 
alternative  uses  could  appear  to  be  the  cause  of  the  decline  in  ducks  when  they  are  in  fact  the  effect. 

In  more  traditional  environmental  risk  analysis,  i.e.,  the  analysis  of  human  health  risks,  extrapolation 
is  a  major  problem.  Analysts  must  extrapolate  the  effects  of  low  dosage  exposure  to  a  contaminant  from  data 
on  high  dose  exposures.  To  complicate  matters,  the  extrapolations  must  be  made  from  laboratory  animals 
to  humans.  Dredging  studies  consider  similar  issues  when  estimating  the  lethal  doses  that  will  kill  50% 
(LD50)  of  the  fish  exposed  to  a  contaminant.  To  the  extent  that  the  EPA  risk  analysis  paradigm  may  be 
adapted  for  Corps  studies,  for  either  environmental  or  ecological  risk  analyses,  extrapolation  may  become 
an  even  greater  problem. 

Identifying  extrapolation  from  the  known  to  the  unknown  as  a  source  of  uncertainty  is,  in  some 
respects,  a  cheap  shot.  When  planners  are  forced  to  confront  and  deal  with  the  unknown  what  else  can  they 
do  but  extrapolate?  Given  they  have  no  choices,  is  it  fair  to  criticize  this  method?  Certainly  not,  if  the  sole 
intent  is  criticism. 

Extrapolation,  under  the  best  circumstances,  is  stepping  tentatively  from  the  known  to  the  unknown. 
Recognition  of  the  uses  of  extrapolation  is  the  most  important  first  step  in  addressing  the  uncertainty  that 
accompanies  extrapolation.  Frequently,  planners  extrapolate  without  recognizing  they  have  passed  from  the 
known  to  the  unknown.  Little  if  any  distinction  is  made  between  scientifically  based  facts  and  conjecture 
or  speculation.  Planners  should  recognize  when  they  are  extrapolating  and  say  that  is  how  they  got  from 
point  A  to  point  B. 

Sensitivity  analysis  is,  again,  one  of  the  most  useful  ways  of  resolving  this  problem.  If  we  are  forced 
to  speculate  and  conjecture  as  we  step  from  the  known  to  the  unknown,  surely  we  can  speculate  on 
alternative  outcomes  of  this  step.  When  extrapolation  is  critical  to  study  results  it  is  advisable  to  first  extend 
the  limits  of  what  is  known.  If  additional  data  or  analysis  can  reduce  the  realm  of  the  unknown  and, 
consequently,  the  uncertainty  attending  our  extrapolation,  getting  the  data  and  doing  the  analysis  should 
always  be  the  first  step.  When  extrapolation  is  the  best  approach  to  a  problem,  it  should  be  done  openly  and 
accompanied  by  sensitivity  analysis  when  the  results  of  the  extrapolation  could  affect  project  performance 
or  plan  selection. 


MODELS 

Reality  is  complicated.  Ecosystems  are  too  complex  to  consider  comprehensively  in  an  effective 
manner.  Models  are  used  to  get  around  these  problems.  Models  are  simplified  representations  of  real 
systems  that  capture  the  most  important  elements  of  the  system  we  want  to  understand. 

Consider  a  map.  If  you  want  to  tell  someone  how  to  get  from  one  place  to  another  you  do  not  need 
a  life  size  map  that  details  every  landmark  and  street  encountered  along  the  way.  State  maps  show  interstate 
highways  and  major  roads  only.  If  you  draw  a  map  for  a  friend  it  generally  shows  only  the  route  you  want 
her  to  follow,  with  occasional  cross  streets  for  reference.  When  giving  directions  you  want  people  to  have 
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only  the  information  they’ll  need  to  arrive  at  their  destination.  All  else  is  extraneous  detail.  A  map  is  a 
model  of  a  much  more  complex  and  detailed  real  world. 

Just  as  maps  leave  out  most  of  the  details  and  ignore  a  great  deal  of  information,  so  do  the  models 
used  by  planners.  Model  uncertainty  has  been  discussed  previously  and  will  be  discussed  in  the  next  section 
as  well.  Nonetheless,  model  uncertainty  presents  a  potentially  significant  source  of  uncertainty. 

Models  come  in  many  forms.  The  Hydrologic  Engineering  Center’s  (HEC)  models  are  well  known 
by  the  Corps  and  around  the  world.  The  habitat  evaluation  process  has  spawned  hundreds  of  HSI  models. 
The  Corps  has  developed  a  cost  effectiveness  and  incremental  cost  analysis  model  for  use  in  evaluating 
environmental  projects.  Dozens  of  spreadsheet  models  may  be  developed  to  perform  specific  calculations 
during  any  one  study.  Expert  opinions  may  comprise  models.  The  six-step  planning  process  used  by  the 
Corps  is  itself  a  model.  Models  abound  in  the  environmental  restoration  planning  process. 

The  responsibility  of  the  planner  who  wants  to  deal  effectively  with  uncertainty  is  to  be  aware  of  the 
assumptions  underlying  the  models  he  uses  and  their  limits.  If  important  details  are  omitted  from  the  model 
you  use  you’re  not  likely  to  arrive  at  the  proper  destination.  It  makes  no  difference  whether  the  model  is  an 
HSI  model,  HEC-2,  or  a  simple  spreadsheet  computation  you  developed;  if  it  lacks  important  details  it  could 
lead  you  to  the  wrong  conclusion. 

Many  environmental  restoration  projects  are  conducted  with  existing  data  on  limited  budgets  using 
available  tools.  These  are  circumstances  that  can  lead  to  the  use  of  models  that  are  not  ideally  suited  for  the 
analysis.  When  this  is  the  case,  and  it  may  often  have  to  be,  risk  analysis  may  help  compensate  for  the  lack 
of  detailed  information. 

Most  HSI  models  are  predicated  upon  an  extensive  review  of  the  relevant  literature.  Conflicting 
views  are  sometimes  averaged,  other  times  ignored.  Some  suitability  graphs  are  based  on  scientific 
evidence,  many  others  are  straight  lines  running  from  an  SI  of  0  and  percentage  of  the  habitat  variable  of  100 
through  an  SI  of  1  and  a  percentage  of  the  variable  of  0,  or  vice  versa,  as  shown  in  Figure  1 1 .  The  quality 
of  the  input  data  is  usually  an  important  constraint  in  any  habitat  evaluation.  The  following  section  takes 
a  closer  look  at  the  potential  uncertainty  in  the  HEP  analysis. 

HABITAT  EVALUATION  PROCEDURE 

Planners  are  interested  in  the  restoration  and  protection  of  ecological  resources.  The  viability  of 
plant  and  animal  populations  is,  however,  difficult  to  quantify.  Populations  are  generally  assumed  to  be  a 
function  of  the  site’s  carrying  capacity.  Carrying  capacity,  in  turn,  is  generally  assumed  to  be  a  function  of 
the  quantity  and  quality  of  available  habitat.  Biological  models  linking  levels  of  habitat  quality  and 
observable  characteristics  of  the  ecosystem  of  interest  are  a  popular  tool  for  estimating  the  effectiveness  of 
a  variety  of  management  measures. 
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At  least  14  different  techniques  and  models^^  have  been  developed  to  assist  planners  in  the  task  of 
measuring  plan  effects  on  ecological  resources.  In  general,  these  models  rely  on  quantitative  relationships 
among  habitat  variables  such  as  depth  of  water,  height  of  vegetation,  percentage  of  submerged  substrate  and 
the  like.  Environmental  restoration  plans  consist  of  management  measures,  either  features  or  activities,  that 
are  expected  to  produce  changes  in  the  habitat  variables.  These  changes  in  variables,  in  turn,  translate  into 
changes  in  habitat  quality. 

This  section  uses  the  U.S.  Fish  and  Wildlife  Service’s  (USFWS)  habitat  evaluation  procedures, 
known  as  HEP,  as  the  basis  for  its  discussion.  The  HEP  is  one  of  the  more  widespread  modeling  procedures. 
The  USFWS  has  developed  a  series  of  Ecological  Services  Manuals  (ESM’s)  that  provide  considerable  detail 
in  the  background  and  development  of  the  habitat  evaluation  procedures.  101  ESM  Habitat  as  a  Basis  for 
Environmental  Assessment,  102  ESM  Habitat  Evaluation  Procedures  (HEP),  103  ESM  Standards  for  the 
Development  of  Habitat  Suitability  Index  Models,  and  104  ESM  Human  Use  and  Economic  Evaluation 
(HUEE)  provide  the  best  available  discussion  of  the  HEP  process.  This  discussion  approaches  the  HEP 
process  from  the  user’s  perspective  rather  than  from  the  perspective  of  an  analyst  developing  a  new  HSI 
model. 


In  order  to  make  the  linkage  between  plan  effects  and  changes  in  ecological  resources,  the  typical 
HEP  analysis  identifies  a  single  species  to  serve  as  the  target  species.  The  underlying  presumption  is  that 
if  this  target  species  is  doing  well,  most  other  species  in  that  ecosystem  will  likewise  do  well.  In  order  to 
quantify  existing  and  future  habitat  quality  for  the  target  species,  a  habitat  suitability  index  (HSI)  model  is 


These  models  are  identified  in  Appendix  C  of  EC  \  \05-2-2\{)  Ecosystem  Restoration  in  the  Civil  Works  Program, 

1  June  1995. 
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frequently  used.  These  models  may  be  developed  specifically  for  a  study  or  they  may  be  USFWS  models 
developed  for  general  use  for  specific  target  species. 

The  HSI  models  identify  the  habitat  variables  that  are  most  critical  to  determining  the  quality  of  the 
target  species’  habitat.  In  addition,  they  describe  the  nature  of  the  mathematical  relationships  among  those 
variables  based  on  informed  professional  judgments,  a  potentially  significant  source  of  model  uncertainties. 
The  models  provide  a  framework  for  describing  the  overall  habitat  quality  through  the  estimation  and 
measurement  of  habitat  variables  and  the  calculation  of  a  variety  of  model  components. 

There  is  considerable  uncertainty  inherent  in  this  evaluation  process.  The  section  that  follows 
Identifies  some  of  the  potentially  more  significant  sources  of  uncertainty.  The  subsequent  section  presents 
an  example  risk  analysis  of  a  hypothetical  HEP  analysis  to  demonstrate  the  feasibility  of  incorporating  basic 
principles  of  risk  and  uncertainty  analysis  into  the  habitat  evaluation  procedures. 

POTENTIAL  SOURCES  OF  UNCERTAINTY  IN  HEP  ANALYSIS 

The  HSI  model  is  one  of  the  principal  determinants  of  the  change  in  habitat  units  that  results  from 
an  environmental  restoration  plan,  the  other  being  the  area  of  habitat  affected  by  the  plan.  The  HSI  model 
is  the  focus  of  the  discussion  that  follows. 

There  are  four  categories  of  uncertainty  encountered  in  a  typical  HSI  model.  First,  there  are 
theoretical  uncertainties.  These  include  the  basic  premise  of  the  HEP  procedure  and  the  model  uncertainties 
discussed  in  previous  sections.  The  existence  of  such  uncertainties  are  acknowledged  but  not  further 
addressed  in  this  report.  Second,  there  is  uncertainty  inherent  in  the  “hard  wiring”  of  the  model,  i.e.,  the 
relationships  field  analysts  use  without  question,  based  on  the  literature  review  and  professional  judgments 
of  the  model’s  creators.  Third,  there  are  uncertainties  attending  measurements  of  model  inputs  by  field 
personnel.  Finally,  there  are  uncertainties  about  project  performance  and  future  variable  values  without  and 
with  a  project. 

This  section  concentrates  primarily  on  the  uncertainty  accompanying  habitat  variables  for  existing 
conditions  as  well  as  future  conditions  without  and  with  a  project ,  the  third  and  fourth  categories  of 
uncertainty.  Using  the  Morgan  and  Henrion  approach  to  uncertainty  analysis,  the  following  subsections 
identify  the  type  of  quantity  that  is  uncertain  as  well  as  the  source  of  uncertainty  and  potential  methods  for 
addressing  this  type  of  uncertainty.  This  information  is  summarized  in  a  table  at  the  end  of  the  section. 

DELINEATION  OF  THE  STUDY  AREA 

The  HEP  analysis  study  area  is  defined  by  the  USFWS  as  “. . .  those  areas  where  biological  changes 
related  to  the  land  or  water  use  proposal  under  study  are  expected  to  occur.”  This  would  include  areas 
affected  directly  or  indirectly.  Uncertainties  can  arise  because  project  effects  can  involve  complex  biological 
linkages.  Alternatively,  privately  owned  lands  that  may  be  affected  by  a  project  are  sometimes  inaccessible 
to  planners  and  may  be  excluded  from  the  project  area  simply  because  they  cannot  be  adequately 
investigated. 
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Simply  identifying  the  area  affected  by  a  project  is  often  an  exercise  in  uncertainty.  The  planning 
area  is  a  domain  variable.  The  source  of  the  uncertainty  is  approximation.  Because  this  is  a  model  domain 
parameter  to  be  determined  by  project  analysts  it  would  be  most  appropriate  to  address  the  uncertainty,  if 
it  is  judged  significant,  through  sensitivity  analysis.  That  is,  alternative  definitions  of  the  study  area  can  be 
analyzed  to  determine  whether  or  not  the  recommended  plan  or  its  impacts  are  significantly  affected  by  the 
definition  of  the  study  area.  A  more  comprehensive  definition  of  the  study  area  could  result  in  larger  project- 
related  changes  in  HU’s  and,  consequently,  lower  incremental  costs  per  HU.  This  is  not  likely  to  be  a  serious 
source  of  uncertainty  in  most  instances.  The  default  strategy  would  be  to  select  the  more  encompassing  area 
as  the  study  area. 

TARGET  SPECIES 

Formulation  of  plans  intended  to  benefit  a  single  target  species  could  result  in  unintended  adverse 
effects  on  other  species.  An  HEP  assessment  is  directly  applicable  only  to  the  evaluation  species.  The 
degree  to  which  results  of  an  HEP  analysis  can  be  extrapolated  to  the  larger  wildlife  community  depends  on 
the  care  taken  in  selecting  the  evaluation  species.  No  one  target  species  can  adequately  represent  the  welfare 
of  all  ecological  resources  in  any  but  the  simplest  of  ecosystems.  Even  species  in  the  same  guild  can  have 
very  different  responses  to  certain  changes  in  the  habitat  variables. 

During  the  early  years  of  habitat  evaluation,  identification  of  a  single  target  species  was  considered 
a  giant  step  forward.  As  experience  in  developing  and  applying  models  has  been  gained,  there  is  a  growing 
realization  that  what  was  learned  about  target  species  modeling  can  be  applied  to  communities  as  well. 
Many  biologists  contend  the  best  HSI  models  would  be  community-level  habitat  evaluation  models.^^  If  that 
is  not  feasible,  using  a  suite  of  evaluation/target  species  may  better  balance  the  diverse  habitat  needs  of  a 
community. 

In  truth,  evaluation  species  are  often  selected  with  far  less  analysis  and  for  far  more  pragmatic 
reasons.  For  example,  advisory  groups  may  have  identified  key  species  for  previous  studies  or  other 
purposes.  Adopting  one  of  them  gains  instant  credibility  from  these  groups  even  if  the  selections  were  not 
optimal  for  the  specific  study  area.  In  other  cases,  non-Federal  partners  or  other  powerful  stakeholders  may 
identify  the  evaluation  species.  The  less  scientific  the  selection  process  the  more  important  it  becomes  to 
do  a  sensitivity  analysis,  using  a  species  that  more  effectively  indicates  the  ecosystem  function. 

The  target  species  is  a  decision  variable.  The  associated  uncertainty  arises  primarily  from  subjective 
judgment  but  there  is  potential  for  professional  disagreement.  Given  the  complexity  of  ecosystems  it  may 
be  impossible  to  identify  a  single  target  species  that  represents  the  full  range  of  potential  ecological  effects. 
Yet,  study  budgets  may  make  community-level  models  or  suites  of  species  impractical  choices.  In  this  case 
the  analyst  picks  the  model  that  is  most  practical  after  weighing  the  analytical  requirements  of  the  study,  its 
budget  and  other  constraints. 


See  “Guidelines  for  the  Development  of  Community-level  Habitat  Evaluation  Models”  by  Richard  L.  Schroeder  and  Sandra  L. 
Haire,  Biological  Report  8  dated  February  1993  for  the  USFWS. 
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A  common  source  of  disagreement  can  arise  from  the  fact  that  there  are  at  least  two  approaches  to 
selecting  the  evaluation  species.  One  is  to  pick  a  species  with  high  public  interest.  Another  is  to  pick  a 
species  that  provides  a  broad  ecological  perspective  of  the  area.  A  species  of  interest  to  hunters  or  fishers 
may  be  a  pragmatic  choice  but  it  may  not  be  a  good  indicator  of  the  ecosystem’s  well-being.  Disagreement 
over  the  objectives  of  the  study  could  lead  to  disagreement  over  the  evaluation  species  as  well. 

The  most  appropriate  treatment  of  uncertain  decision  variables  from  either  of  these  two  causes  is 
to  use  a  sensitivity  analysis.  That  is,  try  other  species  and  see  if  the  plans  would  change.  Given  the  potential 
sensitivity  of  habitat  quality  to  key  habitat  variables  and  the  sensitivity  of  these  variables  to  management 
measures  it  is  not  difficult  to  imagine  that  plans  could  change  significantly  in  some  cases  if  different  target 
species  are  used. 

This  analysis  of  different  species  can  be  done  as  a  pure  sensitivity  analysis  or  it  could  be  more 
appropriately  done  in  the  evaluation  of  a  suite  of  target  species.  It  is  not  likely  that  such  a  step  would  be 
possible  for  most  small  budget  studies.  However,  in  larger  studies  suites  and  sensitivity  analyses  may  be 
appropriate. 

HSI  MODELS 

Model  uncertainty  is  not  the  focus  of  this  discussion,  but  it  is  important  enough  that  to  ignore  it 
would  be  negligent.  The  HEP  analysis  and  the  HSI  models  that  are  used  in  the  HEP  are  all  the  result  of 
informed  professional  judgment.  Nevertheless,  they  are  subjective.  They  are  scientific  guesswork. 

There  are  some  who  would  challenge  the  assumption  that  habitat  quality  is  a  good  indicator  of  the 
viability  of  ecological  resources,  opting  instead  for  condition  or  diversity  indices,  studies  of  energy  flow  and 
so  on.  There  may  be  a  tenuous  relationship  between  carrying  capacity  and  habitat  quality  as  measured  by 
the  variables.  The  problem  of  extrapolating  assumed  impacts  on  the  habitat  of  a  target  species  to  the  well 
being  of  an  entire  ecosystem  has  already  been  touched  upon. 

There  are  still  more  specific  sources  of  uncertainty.  Habitat  models  in  the  HEP  analysis  must  be  in 
index  form.  The  index  runs  from  0  to  1.0  and  is  dimensionless.  An  index  of  0  indicates  a  complete  lack  of 
suitability  of  any  area  as  habitat  for  the  indicator  species.  An  HSI  of  1  identifies  the  area  as  an  ideal  habitat, 
in  a  practical  sense.  The  HSI  is  defined  as: 

Study  Area  Habitat  Conditions 

HS1=  _ 

Optimum  Habitat  Conditions 

Furthermore,  the  use  of  an  HSI  model  in  an  HEP  process  requires  the  HSI  to  be  a  linear  index.  That  is,  an 
improvement  from  0.1  to  0.2  is  the  same  magnitude  as  an  improvement  from  0.8  to  0.9.  It  requires  a 
heroically  optimistic  leap  of  faith  to  expect  the  relationship  between  HSI  and  carrying  capacity  to  be  linear. 
Linearity  may  not  be  a  realistic  assumption  but  it  does  allow  us  to  proceed. 
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The  HSI  is  an  index  variable  that  purports  to  relate  habitat  and  carrying  capacity.  Many  users  of  the 
HEP  may  not  understand  its  assumptions  and  limitations.  Uncertainty  may  arise  from  imprecise  language 
that  results  in  less  than  a  clear  understanding  of  what  an  HEP  or  an  HSI  is  to  do.  The  most  effective 
treatments  for  this  are  education  and  training. 

The  structure  of  the  models,  i.e.  the  “hard  wiring”,  is  another  important  source  of  potential  model 
uncertainty.  Not  all  habitat  variables  are  of  equal  importance  to  the  quality  of  the  target  species’  habitat. 
Assumptions  about  the  relative  importance,  the  additivity,  and  compensation  among  variables  are  built  into 
the  mathematical  assumptions  of  the  model.  For  example,  the  relationships  among  variables  that  determine 
a  particular  life  requisite  might  be  modeled  as  follows: 

HSI  =  lesser  of  V,  or  [(V^  V.)  ^  +  .5(V4  Vs)-^] 

This  formula  yields  an  analytical  result.  It  is  nothing  more  than  an  attempt  to  reflect  the  relative 
importance  of  different  variables  in  determining  the  quality  of  a  habitat.  It  gives  a  predominate  role  to  the 
limiting  factor  V,.  The  weights  assigned  to  the  other  four  variables  by  the  formula  are  very  subjective. 
Suppose  we  used  a  cube  root  instead  of  a  square  root?  Suppose  the  variables  were  additive  rather  than 
multiplicative?  Should  the  weight  for  the  (V4  ^  factor  be  .25  or  .75  instead  of  .5?  Would  these  or  any 

other  changes  have  an  effect,  significant  or  otherwise,  on  the  evaluated  effects  of  the  plan  or  on  plan 
formulation? 

In  addition  to  concerns  about  the  extent  to  which  the  models  reflect  the  workings  of  ecosystems,  we 
need  to  be  concerned  about  how  people  learn  the  HSI  procedures.  Are  they  all  trained  and  certified  by 
USFWS  or  do  they  learn  it  by  looking  at  previous  reports,  talking  to  people  who  have  done  them  before,  or 
winging  it?  Expert  opinion,  the  development  of  special  models,  and  peer  review  may  be  methods  for 
improving  the  structure  of  HSI  models. 

As  good  as  these  new  biological  models  are,  they  are  still  only  models  and  are  subject  to  potential 
model  uncertainty.  As  noted  earlier,  model  uncertainties  may  be  the  most  significant  and  the  most  difficult 
to  deal  with.  Model  uncertainties,  as  important  as  they  may  be,  are  not  the  uncertainties  that  Corps  planners 
should  deal  with  first.  They  are  mentioned  here,  however,  because  they  may,  in  the  long-run,  prove  to  be 
among  the  more  important  sources  of  uncertainty  and  they  should  not  be  forgotten. 

HABITAT  VARIABLE  MEASUREMENTS 

Estimating  the  existing  values  of  habitat  variables  is  perhaps  the  most  basic  source  of  uncertainty 
in  the  HEP  analysis.  Many  environmental  restoration  projects  are  small  budget  planning  studies.  Although 
the  HSI  models  usually  provide  recommendations  on  the  collection  of  data.  Section  1135  studies,  for 
example,  are  intended  to  be  based  as  much  as  possible  on  existing  data.  The  HEP  analyses  for  these  projects 
are  often  subjective. 

Typically,  data  must  be  collected  on  variables  that  define  the  species’  life  requisites.  These  include 
cover  type,  food,  and  other  factors.  Baseline  habitat  data  will  generally  be  based  on  subjective  estimates 
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(opinion),  primary  data  (field  measurement)  and  secondary  data  (e.g.,  remote  sensing).  For  example, 
estimates  of  the  percentage  of  unsubmerged  substrate,  the  percentage  of  land  in  the  affected  area,  the  height 
of  vegetation  or  the  depth  of  water  may  be  based  on  conversations  with  professionals  familiar  with  the  study 
area  or  on  very  cursory  eyeball  estimates.  These  estimates  carry  the  weight  of  informed  professional 
opinion.  It  is  a  matter  of  convenience  that  we  not  challenge  these  numbers  because  to  do  so  would  increase 
study  budgets  and  analytical  requirements.  Or  would  they? 

As  the  example  in  the  section  that  follows  shows,  it  is  a  relatively  easy  matter  to  increase  the 
accuracy  of  one’s  estimate  of  the  value  of  habitat  variables  by  using  interval  estimates  rather  than  point 
estimates.  Although  point  estimates  are  more  precise,  they  are  never  more  accurate.  If  an  analyst’s  job 
depended  on  being  right,  who  would  prefer  to  say  the  average  depth  of  water  is  30  cm  to  saying  it  is  between 
25  and  35  cm? 

When  an  analyst  is  not  sure  of  the  exact  value  of  a  variable  it  is  just  as  easy  to  have  him  estimate  it 
as  an  interval  as  a  point.  That  is  no  extra  work.  In  fact,  going  from  an  uncertain  interval  over  which  he  feels 
fairly  comfortable  in  his  accuracy  to  a  seemingly  certain  single  point,  is  often  the  more  difficult  task. 

When  study  budgets  are  sufficient  or  when  the  problems  are  significant  enough,  planners  may 
actually  take  measurements  of  variables.  This  will  invariably  be  a  sampling  process.  It  may  be  done  through 
remote  sensing  techniques,  using  geographic  information  system  databases  or  it  may  be  direct  field 
measurements.  When  sampling  is  used,  inferential  statistical  techniques  can  be  used  to  estimate  the 
parameters  of  the  sampling  distribution.  The  sample  data  are  used  to  estimate  a  sample  mean  and  a  standard 
error.  These  values  can  then  be  used  to  define  a  normally  distributed  sampling  distribution  or  confidence 
intervals.  These  results  can  be  used  for  subsequent  risk  analyses. 

The  habitat  variable  quantities  are  empirical  quantities.  They  are  uncertain  because  of  measurement 
error,  statistical  variation,  subjective  judgement,  disagreement,  variability,  inherent  randomness  or 
approximation.  The  treatments  of  such  uncertainties  span  the  range  of  analytical  techniques  discussed  earlier 
in  the  manual. 

EXISTING  HABITAT  UNIT  ESTIMATES 

An  HEP  analysis  is  structured  around  the  calculation  of  habitat  units  for  each  evaluation  or  target 
species  in  the  study  area.  HU’s  are  defined  as  the  product  of  the  Habitat  Suitability  Index  (quality)  and  the 
total  area  of  habitat  (quantity). 

The  HSI  for  available  habitat  is  a  function  of  the  suitability  of  all  the  evaluation  species’  life 
requisites  in  the  study  area.  Every  model  has  specific  data  requirements  and  a  unique  mathematical 
relationship  for  combining  the  habitat  variables  into  an  HSI.  The  HSI  for  available  habitat  can  embody  many 
different  and  cumulative  sources  of  uncertainty.  These  include  uncertainties  about  the  representativeness 
of  the  target  species  selection,  uncertainty  about  the  habitat  variables  as  well  as  model  uncertainty  about  the 
manner  in  which  an  HSI  is  calculated. 
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Ideally,  it  may  be  desirable  to  develop  different  HSI’s  for  sections  of  the  study  area  over  which  the 
quality  or  quantity  of  the  habitat  variables  vary  significantly.  It  is  common  practice,  however,  to  calculate 
a  single  HSI.  Although  such  an  HSI  should  represent  average  conditions  over  the  study  area,  it  is  rarely  clear 
from  the  study  reports  whether  that  is  the  case  or  not.  Thus,  there  is  a  fundamental  question  about  what  the 
calculated  HSI  represents.  Is  it  the  average  HSI?  Is  it  representative  of  the  entire  study  area  or  a  small  part 
of  it? 


In  addition,  there  may  be  some  uncertainty  about  the  amount  of  land  that  will  be  affected  by  the 
proposed  project.  This  uncertainty  could  arise  from  measurement  errors  (e.g.,  planimetering  inexact  areas 
on  maps  of  questionable  accuracy)  or  basic  uncertainty  about  the  spatial  extent  of  project  effects. 

The  estimate  of  existing  HU’s  is  an  outcome  variable  and  its  uncertainty  is  derived  from  the  various 
sources  of  uncertainty  that  affect  the  quantities  that  determine  HU’s.  The  treatments  of  these  uncertainties 
span  the  gamut  of  known  methods. 

FUTURE  HABITAT  VARIABLE  ESTIMATES 

Measurements  of  existing  habitat  variables  are  theoretically  determinate.  Given  enough  time  and 
money,  analysts  could  go  out  into  the  field  and  take  careful  measurements  that  would  enable  them  to 
quantity  habitat  variables  with  a  high  degree  of  certainty.  Estimates  of  future  habitat  variable  values  are  by 
their  nature  uncertain.  There  is  no  way  these  values  can  be  certain  no  matter  how  much  time  or  money  the 
investigators  have.  Forecasting  the  future  is  fundamentally  uncertain. 

At  least  two  estimates  of  future  habitat  variable  values  must  be  prepared.  One  describes  what  will 
most  likely  happen  to  the  habitat  variables  if  no  project  is  implemented.  The  other  describes  what  will  most 
likely  happen  if  a  project  is  implemented.  There  must  be  a  with  project  future  forecast  for  every  alternative 
plan  evaluated.  It  seems  a  matter  of  basic  honesty  in  the  analysis  to  acknowledge  the  uncertainty  inherent 
in  this  aspect  of  the  HEP  analysis. 

These  values  are  chance  variable  quantities  (i.e.,  empirical  quantities)  and  their  sources  of 
uncertainty  include  subjective  judgment,  professional  disagreement,  statistical  variation,  variability,  and 
inherent  randomness.  Treatments  of  the  uncertainty  could  include  most  known  methods. 

CHANGE  IN  HU’S 

Ostensibly,  the  purpose  of  the  HEP  analysis  is  to  calculate  changes  in  habitat  units  that  are 
attributable  to  project  effects.  An  increase  in  HU’s  is  one  measure  of  the  presumed  outputs  of  the  project. 
The  change  in  HU’s  is,  in  Morgan  and  Henrion’s  language  of  uncertainty,  an  outcome  variable.  It  is  an 
uncertain  value  if  any  of  the  models  or  values  required  to  estimate  it  are  uncertain.  If  there  is  model 
uncertainty,  uncertainty  about  the  spatial  extent  of  project  effects,  uncertainty  about  the  choice  of  evaluation 
species,  uncertainty  in  the  measurement  of  variables,  and  so  on,  then  there  is  also  uncertainty  in  the 
measurement  of  project  outputs  as  evidenced  through  the  change  in  habitat  units  from  the  without  project 
to  the  with  project  condition. 
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The  timing  of  changes  in  habitat  variables  can  ultimately  have  significant  effects  on  the  change  in 
HU’s  for  a  study  area.  That  is,  do  the  forecast  changes  in  habitat  variables  take  place  instantaneously  or  over 
many  years?  Devoting  more  attention  and  detail  to  estimating  the  temporal  sequence  of  changes  in  habitat 
units  can  be  another  effective  way  of  dealing  with  a  potentially  significant  source  of  uncertainty. 

PROJECT  PERFORMANCE  IN  HEP  ANALYSIS 

A  Corps  biologist  consulted  during  the  preparation  of  this  manual  pointed  out  that  the  public  often 
knows  what  it  wants  but  translating  that  into  terms  an  engineer  can  deal  with  is  difficult.  He  aptly  described 
the  challenge  when  he  said,  “Land  managers  talk  function  and  feathers  while  engineers  talk  feet  and  inches.” 
Nonetheless,  he  pointed  out  that  when  a  community  has  the  resources  and  facilities  of  the  Corps  of  Engineers 
to  do  good  things  for  the  environment,  that  is  an  advantage  most  environmentalists  will  never  have. 

Two  uncertain  issues  arise  regarding  environmental  projects.  One  is,  do  we  have  the  right  project? 
The  other  is,  will  it  work?  The  first  issue  requires  us  to  ask  the  question,  can  this  project  contribute  to  our 
planning  objectives?  The  follow-up  question  is,  will  it?  During  the  preparation  of  this  manual  a  surprising 
degree  of  frankness  was  encountered  among  Corps  personnel  concerning  the  basic  uncertainty  about  the 
performance  of  environmental  projects. 

The  environmental  restoration  programs  are  new.  The  ecosystems  are  complex.  And,  although  the 
science  is  often  state-of-the-art,  it  can  only  take  the  analysts  so  far,  then  they  must  guess  and  hope.  How 
long  will  the  geotubes  last  in  the  water?  Will  the  coconut  fiber  grass  mats  take  root?  Will  the  plants  that 
grow  in  the  restored  wetlands  provide  a  diversity  of  food  and  cover  for  species  or  will  invasive  plants  with 
little  or  no  food  and  cover  values  grow?  Will  the  closure  structure  allow  the  marsh  to  drain  in  time  to 
prevent  disruption  of  salinities  and  nests  in  the  normally  unsubmerged  substrate?  The  performance  questions 
go  on  and  on. 

Even  though  virtually  every  analyst  consulted  could  raise  questions  about  the  performance  of  their 
projects,  none  of  the  project  reports  reviewed  reflected  this  uncertainty.  That  is  not  surprising.  Admitting 
a  project  could  have  an  outcome  other  than  the  one  the  partners  hope  for  does  nothing  to  build  support  for 
the  project.  Until  data  on  project  performance  begin  to  become  available  through  monitoring  programs,  there 
is  no  objective  way  to  know  how  serious  a  problem  this  may  be,  if  it  is  a  problem  at  all. 

At  this  point,  we  simply  note  that  project  performance  is  a  potentially  significant  source  of 
uncertainty.  It  is,  in  fact,  an  empirical  quantity,  a  chance  variable,  but  it  is  estimated  as  an  outcome 
variable.  Defining  alternative  with  project  conditions  that  reflect  different  outcomes  and  evaluating  the 
changes  in  habitat  units  under  each  is  one  feasible  way  to  address  this  problem. 

Table  5  summarizes  the  uncertainties  described  in  this  section.  They  are  precisely  the  kinds  of 
uncertainty  that  would  be  addressed  in  the  example  that  follows,  although  emphasis  in  the  example  is  on  the 
last  two  items  in  the  table. 
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Table  5:  HEP  Uncertainties 

Item 

Type  of  Quantity 

Source  of  Uncertainty 

Treatment  of  Uncertainty 

Study  area 

Model  domain 

Approximation 

Sensitivity  analysis 

Evaluation 

species 

Decision 

Disagreement,  statistical 
variation,  subjective  judgment 

Sensitivity  analysis 

Total  area 

Empirical 

Approximation,  statistical 
variation,  subjective  judgment, 
variability,  disagreement 

Sensitivity  analysis,  probability 
distribution,  sampling 
distribution,  uncertainty  criteria, 
expert  opinion 

Habitat  variable 

measurements 

Empirical,  decision, 
index 

Approximation,  statistical 
variation,  subjective  judgment, 
variability,  linguistic 
imprecision,  disagreement 

Education  &  training,  sensitivity 
analysis,  probability 
distribution,  sampling 
distribution,  uncertainty  criteria, 
expert  opinion 

HU’s 

Outcome 

Approximation,  statistical 
variation,  subjective  judgment, 
variability,  linguistic 
imprecision,  disagreement 

Education  &  training,  sensitivity 
analysis,  probability 
distribution,  sampling 
distribution,  uncertainty  criteria, 
expert  opinion 

RISK  ANALYSIS  EXAMPLE:  MOTTLED  DUCK  HSI  MODEL 

Is  risk-based  analysis  of  environmental  restoration  projects  feasible?  What  will  it  take  to  do  such 
analyses?  This  section  presents  a  simple  example  of  how  risk-based  analysis  can  be  used  to  estimate  the 
expected  change  in  HU’s.  The  example  uses  the  mottled  duck  HSI  model  to  demonstrate  the  feasibility  of 
risk  analysis  with  commonly  available  basic  information.  Data  from  a  1992  Section  1135  study  were 
modified  and  used  for  this  analysis.  The  subsections  that  follows  describe  the  analysis  and  its  results. 

The  basic  elements  of  the  mottled  duck  model  are  described  below.  This  example  focuses  on  how 
the  uncertainty  in  input  data  can  be  reflected.  No  attempt  is  made  to  address  model  uncertainty,  though  some 
opportunities  for  doing  so  are  discussed.  The  modification  of  the  data  was  arbitrary.  The  results  of  the 
example  analysis  have  no  significance  beyond  the  demonstration.  They  have  absolutely  no  bearing  on  the 
original  study. 

Mottled  Duck  HSI  Model 


The  description  that  follows  is  taken  from  the  September  1983  Habitat  Suitability  Index  Models: 
Mottled  Duck,  FWS/OBS-82/10.52.  The  model  was  prepared  by  James  C.  Rorabaugh  and  Phillip  J.  Zwank 
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of  Louisiana  State  University  for  the  Fish  and  Wildlife  Service  of  the  U.S.  Department  of  the  Interior.  The 
relationship  of  the  habitat  variables  and  life  requisites  to  the  habitat  suitability  index  are  shown  in  Figure  12. 
The  HSI  is  a  function  of  three  life  requisites:  cover,  food  and  other  (disturbance  level).  There  are  eight 
habitat  variables  defined  as  shown  below.  Suitability  graphs  transform  habitat  variable  measurements  into 
suitability  indices  on  a  0  to  1  scale.  The  functional  form  of  their  suitability  graph  is  given  in  parenthesis. 

V,  =  Percentage  of  unsubmerged  substrate  covered  by  rushes,  bullrushes,  or  cattails. 

(Linear  suitability  graph.) 

V2  =  Percentage  canopy  cover  of  trees  and  shrubs  on  unsubmerged  substrate.  (Piecewise 

linear  suitability  graph.) 

V3  =  Structure  of  herbaceous  vegetation  on  unsubmerged  substrate.  (Step  function 
suitability  graph  with  four  classes/steps.) 

V4  =  Percentage  of  continually  submerged  substrate  covered  by  woody  or  herbaceous 
emergent  vegetation.  (Piecewise  linear  suitability  graph.) 

V5  =  Structure  of  woody  or  herbaceous  emergent  vegetation  growing  in  continually 
submerged  substrate.  (Step  function  suitability  graph  with  four  classes/steps.) 
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Vfi  =  Percentage  of  study  area  that  is  land.  (Piecewise  linear  suitability  graph.) 

V7  =  Percentage  of  continually  submerged  substrates  with  water  depth  less  than  30  cm 
at  low  mean  tide.  (Linear  suitability  graph.) 

Vg  =  Disturbance  level;  extreme;  moderate;  minimal;  and,  none.  (Step  function 
suitability  graph.) 

Four  model  components  and  three  life  requisites  are  calculated  using  these  input  variables.  Nesting 
hen  cover  (NHC)  is  the  cube  root  of  the  product  of  the  Si’s  for  V„  V2,  and  V3.  Hen  with  brood  cover  (HBC) 
is  the  square  root  of  the  product  of  the  Si’s  for  V4  and  Vj.  Cover  structure  (CS)  is  defined  as  the  minimum 
ofNHC  or  HBC.  The  cover  ratio  (CR)  is  the  SI  for  V5.  The  reproductive  cover  life  requisite  (C)  is  the  cube 
root  of  the  product  of  CS  squared  and  CR.  The  food  life  requisite  (F)  is  the  SI  for  V7.  The  other  life  requisite 
(O)  is  the  SI  for  Vg.  The  mottled  duck  HSl  is  the  minimum  of  C,  F  or  O. 

To  demonstrate  the  feasibility  of  introducing  simple  risk  analysis  to  the  estimation  of  an  HSI  the 
mottled  duck  model  was  built  in  a  spreadsheet.^'*  Appendix  A  explains  the  spreadsheet  model  in  greater 
detail.  The  original  Corps  analysis  estimated  the  existing  condition  HSI  to  be  0.31  and  the  improved 
condition  HSI  was  0.79.  To  simplify  this  demonstration  a  simple  comparison  of  with  and  without  condition 
HSI’s  is  used.  There  is  no  discounting  or  multi-year  analysis.  In  order  to  allow  the  investigation  of  the 
effect  of  a  risk  analysis  on  potential  incremental  costs,  habitat  units  were  computed  using  the  same  1,000 
acres  of  land  used  in  the  original  Section  1 135  report.  Project  costs  of  $1,824,000  were  used.  Because  a 
single  increment  of  1,000  acres  was  used  in  this  calculation  for  demonstration  purposes,  the  example  should 
not  be  considered  a  true  incremental  cost  analysis.  Although  it  does  reflect  the  approach  of  the  original 
report  a  true  incremental  cost  analysis  requires  different  increments  of  output. 

Using  the  original  without  and  with  project  condition  input  data  from  the  1992  report,  habitat 
variable  values  were  allowed  to  vary  according  to  a  simple  probability  distribution.  A  value  for  each  variable 
was  randomly  sampled  from  this  distribution  using  a  Latin  Hypercube  process  and  these  values  were  then 
used  to  generate  other  model  components,  the  HSI’s  without  and  with  the  project,  and  the  change  in  HU’s 
and  the  incremental  costs  of  those  changes. 

For  example,  V,,  the  percentage  of  unsubmerged  substrate,  was  estimated  in  the  report  to  be  10 
percent  without  or  with  the  project.  In  the  example,  this  value  was  arbitrarily  allowed  to  vary  from  0  to  20 
percent^’.  A  uniform  distribution  was  used  for  simplicity. 


Excel  V.  5.0  with  the  @RISK  add-in  v.  3.1  was  used  to  develop  the  model.  A  more  detailed  explanation  of  the  model  is 
presented  at  Appendix  A, 

25 

Uniform  distributions  were  used.  The  minimum  and  maximum  values  were  obtained  by  adding  and  subtracting  10  percent 
to/from  the  value  used  by  the  District,  except  where  a  logical  constraint  of  0  or  1 00  prevent  this.  These  arbitrary  changes  were  made  for  the 
sake  of  simplicity. 
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It  would  not  be  uncommon  for  an  analyst  working  under  budget  and  time  constraints  to  be  uncertain 
of  the  actual  percentages  and  classes  of  habitat  variable  values.  Although  it  may  be  more  precise  for  the 
analyst  to  say  the  percentage  of  unsubmerged  vegetation  is  1 0  percent  that  is  not  likely  to  be  the  actual  value. 
It  would  most  likely  be  far  more  accurate  to  estimate  this  variable  with  an  interval,  like  0  to  20  percent. 
Though  far  more  sophisticated  analyses  could  be  done,  this  simple  change  serves  the  purpose  of 
demonstrating  the  ease  with  which  interval  input  data  can  be  obtained  and  used. 

The  actual  range  of  values  could  be  estimated  from  confidence  intervals  obtained  from  the  sampling 
distribution  of  a  proportion  (or  mean)  if  sampling  from  aerial  photographs  or  field  measurements  are  used. 
Or,  it  could  be  based  upon  the  opinion  of  knowledgeable  personnel  of  the  resource  management  agency. 
Alternatively,  it  might  be  based  on  the  “eyeball”  estimate  of  the  Corps  biologist.  In  any  case,  it  is  a 
relatively  simple  matter  to  prepare  interval  estimates  of  input  variable  values. 

Rather  than  enter  V,  as  1 0  percent,  the  unsubmerged  substrate  was  estimated  to  range  from  0  percent 
to  20  percent.  A  uniform  distribution  was  assumed  for  all  input  variables  in  this  example.  Table  6 
summarizes  the  without  project  condition  distributions  used  for  the  input  variables.  With  condition 
distributions  were  changed  for  the  variables  the  project  was  expected  to  improve.  The  minimums  and 
maximums  of  the  uniform  distributions  are  presented  as  percentages  in  the  table.  Percentages  of  each  of  the 
four  classes  for  the  step  function  habitat  variables  are  provided.  Classes  that  are  not  listed  are  not  present 
in  the  study  area  and  are  assumed  to  equal  zero  percent.  Thus,  for  V3  there  is  0  to  10  percent  in  Class  3  with 
the  remainder  (90  to  100  percent)  in  Class  4. 


Table  6:  Without  Project  Condition  Distribution  Parameters 

Variable 

Uniform  Distribution  Min  and  Max 

Class  3%  for  V3 

0,  10 

Class  4%  for  V3 

90,  100 

Class  3%  for  V; 

0,  10 

Class  4%  for  V, 

90,  100 

Class  3%  for  Vg 

0,  10 

Class  4%forV, 

90,  100 

V, 

0,  20 

V2 

0,  10 

V4 

0,  20 

V6 

10,  30 

V7 

70,  90 
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Using  the  Terminology 

Early  sections  of  this  report  developed  a  taxonomy  of  uncertainty  that  can  be  used  to  help  an  analyst  to 
think  about  how  to  approach  the  analysis.  Table  5  presented  a  somewhat  generic  example.  Here  you’ll  find  a 
more  specific  application  of  the  concepts  to  the  mottled  duck  model.  The  example  is  a  hypothetical  one, 
loosely  based  on  an  actual  study. 


Item 

Type  of  Quantity 

Source  of  Uncertainty 

Treatment  of  Uncertainty 

Study  Area 

Model  domain  parameter 

Approximation 

Assumed  certain 

Mottled  Duck 

Decision  variable 

Subjective  Judgment 

Assumed  certain 

1,000  Acres 

Empirical  parameter 

Approximation 

Assumed  certain 

V, 

Empirical  parameter 

Variability 

Probability  distribution 

V2 

Empirical  parameter 

Variability 

Probability  distribution 

V3 

Empirical  parameter 

Variability 

Probability  distribution 

V4 

Empirical  parameter 

Variability 

Probability  distribution 

V5 

Empirical  parameter 

Variability 

Probability  distribution 

V* 

Empirical  parameter 

Approximation 

Probability  distribution 

V7 

Empirical  parameter 

Approximation 

Probability  distribution 

Vg 

Empirical  parameter 

Subjective  Judgment 

Probability  distribution 

HSI 

Outcome  criterion 

All  the  above 

Determined  by  inputs 

HU’S 

Outcome  criterion 

AH  the  above 

Determined  by  inputs 

$  per  HU 

Outcome  criterion 

All  the  above 

Determined  by  inputs 

The  first  three  quantities  have  been  assumed  to  be  determinate  values,  i.e.,  the  analyst  considers  them 
to  be  certain  values.  These  items  are  not  treated  as  uncertain  in  this  analysis.  This  was  done  because  of  the 
“politics”  of  the  study.  The  first  five  habitat  variable  values  are  based  on  the  results  of  sample  measurements. 
They  are  uncertain  because  of  the  variability  in  the  sample  results.  The  sixth  and  seventh  habitat  values  were 
based  on  approximations  made  from  aerial  photographs.  The  eighth  variable  value  is  based  on  the  subjective 
judgment  of  the  park  rangers  who  are  most  familiar  with  the  project  area. 

The  treatment  for  the  habitat  variable  uncertainty  is  to  represent  the  potential  range  of  values  as  a 
probability  distribution.  The  last  three  items  are  outputs  of  the  model.  The  uncertainty  in  them  is  a  result  of  the 
uncertainty  in  the  variables  from  which  they  are  calculated. 
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A  Latin  Hypercube  procedure  was  used  to  transform  the  HSI  model  from  a  simple  calculation  to  a 
simulation  model.  The  model  calculated  a  without  project  HSI,  a  with  project  HSI,  HU’s  without  and  with 
the  project,  as  well  as  the  change  in  the  number  of  HU’s  attributable  to  the  project  during  each  iteration  of 
the  model.  The  cost  per  each  additional  HU  was  also  calculated.  One  thousand  iterations  of  the  model  were 
run  in  which  a  value  was  randomly  selected  from  each  input  distribution  and  “plugged”  into  the  HSl 
computation.  Thus,  1,000  pairs  of  HSI  values,  HU’s  and  their  costs  were  separately  computed  based  on  the 
varying  inputs  shown  in  Table  6. 

The  results  of  the  simulation  are  summarized  in  Table  7.  Minimum,  maximum  and  mean  values  are 
presented  for  selected  model  outputs.  For  example,  the  minimum  observed  HSI  without  the  project  was  0 
while  the  maximum  observed  value  was  0.42.  The  average  without  project  HSI  for  the  1,000  calculations 
was  0.23^®A  simple  variation  of  the  input  variables  results  in  a  considerable  range  in  potential  project 
outputs.  The  project  could  produce  as  few  as  282  HU’s  at  an  incremental  cost  of  $6,479  each  (costs  are 
present  values,  not  average  annual  equivalents)  or  as  many  as  750  HU’s  at  $2,432  each.  Figure  13  shows 
a  cumulative  distribution  of  costs  per  habitat  unit.  There  is  a  30%  chance  HU’s  will  cost  less  than  $4,000 
each  and  nearly  a  70%  chance  they’ll  cost  less  than  $5,000  each.  The  mean  number  of  HU’s  obtained  in  this 
1,000  iteration  simulation  was  456  HU’s  at  an  incremental  cost  of  $4,350  each. 


Table  7:  Selected  Simulation  Results  || 

Item 

Minimum 

Mean 

Maximum 

HSI  Without 

0 

.23 

.42 

HSI  With 

.58 

.69 

.75 

HU’s  Without 

0 

232 

417 

HU’s  With 

580 

688 

750 

Change  in  HU’s 

282 

456 

750 

Cost  per  HU 

$2,432 

$4,350 

$6,479 

These  results  make  the  uncertainty  in  project  performance  very  apparent.  The  number  of  HU’s  that 
result  are  estimated  to  be  between  282  and  750  HU’s.  The  incremental  cost  of  these  HU’s  is  likewise 
uncertain.  However,  if  the  maximum  cost  of  $6,479  per  HU  is  reasonable  there  is  little  reason  to  question 
the  wisdom  of  implementing  the  project.  Now,  suppose  for  the  sake  of  this  example  that  $6,000  per  HU  is 
considered  the  maximum  feasible  incremental  cost  for  political,  institutional,  economic  or  other  reasons. 


Despite  the  tendency  to  want  to  reproduce  table  results  from  table  values,  that  generally  cannot  be  done.  For  example,  the 
minimum  change  in  HU’s  is  282  but  the  minimum  number  of  HU’s  with  the  project  less  the  maximum  number  of  HU’s  without  the  project  do 
not  yield  this  value.  That  is  because  these  minimum  and  maximum  values  were  not  observed  during  the  same  iteration  of  the  simulation. 
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Figure  13:  Cumulative  Distribution  Function  for  HU  C 
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The  cumulative  distribution  function  (CDF)  indicates  that  taking  all  the  considered  uncertainty  into  account 
there  is  only  a  10  percent  chance  costs  will  exceed  this  level.  Thus,  we’re  about  90  percent  sure  the  project 
will  result  in  acceptable  costs  per  unit  of  output.  These  are  a  few  examples  of  the  results  of  a  risk  analysis 
that  could  lead  to  a  better  informed  decision  process. 

The  range  in  the  HSl,  HU  and  cost  values  is  due  largely  to  the  structure  of  the  HSI  modeP.  In  this 
case,  once  life  requisite  indices  are  calculated  for  cover,  food,  and  other  (i.e.,  disturbance),  the  minimum  of 
the  three  becomes  the  HSI.  Although  model  uncertainty  is  not  the  focus  of  this  example,  a  simple  sensitivity 
analysis  of  this  HSI  formula  was  conducted  to  demonstrate  the  feasibility  of  such  analyses. 

In  addition  to  examining  the  decision  rule  of  selecting  the  minimum  life  requisite  SI  as  the  HSI,  two 
alternative  decision  formulas  were  investigated  to  determine  the  sensitivity  of  project  outputs  to  the  HSI 
formula.  The  first  was  a  simple  average  of  the  three  Si’s.  The  second  was  the  cube  root  of  the  product  of 
the  three  Si’s.  There  are  substantial  differences  among  the  means,  as  well  as  the  extreme  values  that  result 
from  such  a  sensitivity  analysis  as  Table  8  shows.  “Minimum”  means  the  HSI  is  based  on  the  minimum  of 
the  three  life  requisite  Si’s.  “Average”  means  the  HSI  is  the  average  of  the  three  life  requisite  HSI’s.  “Cube 
Root”  means  the  HSI  is  based  on  the  cube  root  of  the  product  of  the  three  life  requisite  Si’s.  Using  an 
average  of  the  Si’s  leads  to  substantially  higher  HU  costs.  The  cube  root  formula  could  lead  to  cheaper  HU’s 
but,  on  average,  HU  costs  would  be  expected  to  increase. 

This  is  a  single  example  and  generalizations  cannot  be  made  from  it.  It’s  a  hypothetical  example 
that  offers  only  anecdotal  evidence  about  the  mottled  duck  HSI  and  no  evidence  about  other  HSI  models. 
The  example  is  sufficient,  however,  to  make  the  point  that  model  uncertainty  can  be  a  significant  concern 
in  the  estimation  of  HSI’ s. 

There  are  many  points  in  an  HSI  model  where  somewhat  arbitrary  looking  formulas  are  used  to 
weight  variable  Si’s  or  life  requisites  Si’s.  These  formulas  ultimately  determine  the  HSI’s  and  the  change 
in  HU’s. 

Table  8  indicates  that  estimates  of  project  outputs  are  very  sensitive  to  how  life  requisite  Si’s  are 
combined  into  an  HSI.  These  alternatives  were  used  as  an  arbitrary  test  of  the  sensitivity  of  the  HSI  to  the 
decision  rule.  They  were  not  used  to  suggest  the  model  is  incorrect,  only  that  the  results  are  sensitive  to  the 
model’s  structure.  Investigating  the  cumulative  sensitivities  of  HSI’s  to  these  various  formulas  as  well  as 
to  the  suitability  graphs  and  input  data  would  seem  to  be  a  fruitful  area  of  future  risk  analysis  research. 


A  preliminary  test  of  the  risk-based  HSI  model  used  data  provided  in  the  USFWS  description  of  the  mottled  duck  model. 
Varying  the  input  variable  values  in  this  document  as  described  above  led  to  quite  a  different  result.  Because  the  HSI  in  this  model  is  simply 
the  minimum  of  three  life  requisite  Si’s,  a  single  SI  always  determined  the  HSI.  In  an  extreme  case  like  this  an  analyst  need  gather  data  only  on 
a  that  single  constraining  input  variable,  the  one  that  determined  the  controlling  SI  in  this  preliminary  case.  The  point  to  be  drawn  from  this  is 
that  there  is  a  great  deal  that  can  be  learned  about  HSI  models  from  risk  analysis.  Some  of  this  may  be  helpful  in  directing  data  collection  and 
other  strategies. 
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1  Table  8:  Model  Sensitivity  Based  on  Choice  of  HSI  Formula 

Rule 

Minimum 

Mean 

Maximum 

HU’s:  Minimum 

281 

456 

750 

HU’s:  Average 

95 

168 

291 

HU’s:  Cube  Root 

156 

360 

842 

$/HU:  Minimum 

$2,432 

$4,350 

$6,479 

$/HU:  Average 

$6,278 

$11,758 

$19,257 

$/HU:  Cube  Root 

$2,168 

$7,330 

$11,711 

Risk  analysis  has  the  ability  to  indicate  which  variables  are  most  important  in  the  determination  of 
the  HSI’s,  the  HU’s,  or  the  incremental  cost  per  HU.  Figure  14  shows  the  relative  importance  of  variables 
influencing  the  cost  of  HU’s^*.  The  most  important  variable  was  the  percentage  of  submerged  substrate 
under  the  without  project  condition,  which  has  a  strong  positive  influence  on  costs.  The  second  most 
influential  variable  was  the  percentage  of  submerged  substrate  under  with  project  conditions.  Variable  V5 
without  the  project,  V2  with  the  project,  and  V7  witliout  the  project,  are  the  other  influential  variables  shown 
in  the  tornado  graph. 


f  ^  1 

Figure  14:  Regression  Sensitivity  for  $/HU  | 
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Using  cost  per  HU  as  the  dependent  variable,  single  variable  linear  regressions  were  run  using  each  input  variable  as  the 
independent  variable.  The  graph  reports  those  input  variables  with  the  largest  significant  coefficients. 
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Results  like  these  may  be  helpful  in  providing  insight  into  the  workings  of  HSI  models.  It  would 
surely  be  helpful  to  know  which  variables  are  most  important  before  data  are  collected.  Such  information 
could  be  very  useful  in  developing  data  collection  strategies.  Even  after  the  fact  this  information  is  useful. 
It  suggests  that  if  we  want  to  improve  our  estimates  of  HU  costs  it  would  be  best  to  get  careful  estimates  of 
the  percentage  of  submerged  substrate. 

The  example  presented  here  establishes  at  least  three  facts.  First,  it  is  relatively  easy  to  use 
spreadsheet-based  Monte  Carlo/Latin  Hypercube  simulation  to  introduce  risk  analysis  into  the  estimation 
of  HSI’s  and  HU’s  based  on  varying  data  inputs.  Second,  the  results  of  at  least  some  HSI  computations  will 
be  extremely  sensitive  to  the  HSI  model’s  structure.  This  implies  that  HSI  model  uncertainty  may  be 
important  enough  to  warrant  future  investigation.  Third,  the  results  of  risk  analysis  can  help  analysts 
understand  the  relative  importance  of  the  various  uncertain  factors.  This  information  may  be  helpful  in  data 
collection  strategies,  plan  formulation,  and  incremental  cost  estimation. 

The  possibilities  for  introducing  risk  analysis  to  environmental  restoration  projects  does  not  end  with 
this  brief  discussion  of  HSI  models.  For  example,  at  this  writing  a  beta  version  (2.6)  of  ECO-EASY:  Cost 
Effectiveness  and  Incremental  Cost  Analysis  software  is  available  to  Corps  personnel.  It  marks  a  significant 
contribution  to  the  evaluation  of  environmental  restoration  projects.  It  is,  however,  a  deterministic 
evaluation  that  could  possibly  be  modified  to  incorporate  variation  in  the  expected  number  of  habitat  units 
produced  by  a  project  as  well  as  to  reflect  uncertainty  in  project  costs. 

SUMMARY  AND  LOOK  FORWARD 

Four  major  sources  of  uncertainty  in  environmental  restoration  projects  have  been  tentatively 
identified.  They  are  project  performance,  extrapolation,  models,  and  the  habitat  evaluation  procedure.  A 
simple  example  was  presented  illustrating  how  risk-based  analysis  could  be  used  to  address  some  of  the 
uncertainty  inherent  in  the  HEP  analysis  that  is  so  common  to  environmental  restoration  projects.  Selected 
results  from  this  analysis  were  presented  to  demonstrate  the  types  of  outputs  that  can  result  from  a  risk-based 
analysis.  A  few  suggestions  were  offered  to  show  how  these  outputs  might  enlighten  and  improve  the 
decision  process. 

The  report  concludes  with  the  summary  found  in  the  next  chapter. 
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CHAPTER  SEVEN:  SUMMARY  AND  CONCLUSIONS 

First  steps  are  always  frustrating.  It  takes  so  long  to  work  up  the  courage  to  take  one,  then,  once 
taken,  there  are  so  many  places  we’d  like  to  go,  and  so  quickly.  Introducing  risk  and  uncertainty  analysis 
into  the  evaluation  of  environmental  restoration  projects  now  stands  where  introducing  risk  and  uncertainty 
analysis  in  general  stood  with  the  Corps  a  decade  ago.  The  concepts  are  vague.  Illustrative  examples  using 
specific  tools  and  methods  field  personnel  can  use  are  yet  to  be  developed.  The  required  expertise  does  not 
yet  exist  in  many  districts.  The  costs  to  the  study  are  unknown.  The  impacts  on  plan  formulation  and  project 
construction  are  speculative  but  largely  feared.  The  motivation  for  the  change  is  poorly  understood  and 
widely  suspected.  The  people  who  will  have  to  embrace  the  changes  may  be  unconvinced  of  the  need  to  do 
so  and  appear  resistant.  And  few  can  imagine  how  any  of  this  could  possibly  improve  decisions  and 
decision-making. 

To  combat  these,  and  other  attitudes,  two  things  are  absolutely  clear.  First,  any  risk  and  uncertainty 
analysis  must  be  purposeful.  Second,  it  must  be  scaled  to  the  study. 

A  purposeful  risk  and  uncertainty  analysis  will  improve  decision-making.  This  has  always  been  the 
primary  purpose  of  risk  and  uncertainty  analysis.  Risk  analysis,  once  implemented,  will  help  us  to  avoid 
errors.  In  a  world  of  scarce  program  resources  it  is  essential  that  bad  investments  be  avoided.  It  is  equally 
essential  that  good  investments  not  be  overlooked.  Simple  analyses,  such  as  the  example  presented  in  the 
preceding  chapter,  present  a  full  range  of  potential  results  that  provide  decision-makers  with  a  much  more 
complete  picture  of  the  range  of  potential  project  outcomes.  The  true  value  of  such  analysis  will  not  be 
evident  until  the  tools  are  used. 

Risk  and  uncertainty  analysis  will  subject  studies  to  more  scrutiny.  They  will  require  new  analysis 
and  possibly  some  more  data,  but  these  are  reasonable  costs  to  incur  to  improve  decision-making  if  the 
analysis  is  scaled  to  the  study.  Environmental  restoration  analyses,  like  other  Corps  studies,  vary  in  size, 
significance,  priority  and  budget.  It  would  be  inappropriate,  for  example,  to  require  analysts  to  address  all 
the  potential  sources  of  uncertainty  identified  in  this  report  for  any  environmental  restoration  study.  For 
example,  although  model  uncertainty  may  be  a  significant  source  of  uncertainty  in  the  HEP  analysis,  the 
Corps  analyst  does  not  always  have  influence  over  the  development  of  these  models.  To  address  these 
concerns  could  require  inter-agency  cooperation  that  is  well  beyond  the  scope  of  agency  needs  at  this  point. 
When  Corps  analysts  develop  the  HSI  models  it  would  be  prudent  to  consider  model  uncertainty. 

It  is  more  important  to  look  for  things  that  can  be  done  easily  to  improve  decision-making.  Section 
1 135  studies  were  to  be  done  based  largely  on  existing  data.  Habitat  variable  measurements  in  these  studies 
may  be  along  the  lines  of,  “It  looks  like  40  percent  to  me,  how  about  you?”  In  such  cases,  it  would  be 
inappropriate  to  require  detailed  field  measurements  to  limit  the  uncertainty  attending  these  “eyeball” 
estimates,  but  it  would  not  be  unreasonable  to  make  that  subjective  estimate  as  an  interval.  “That’s 
somewhere  between  30  and  45  percent,  don’t  you  think?”  As  the  study’s  scope,  budget,  and  significance 
increase  there  may  be  compelling  reasons  to  increase  the  sophistication  of  the  analyses,  but  only  if  those 
changes  can  be  demonstrated  to  improve  the  quality  and  timing  of  decision-making. 
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This  report  has  attempted  to  advance  the  discussion  of  introducing  risk  and  uncertainty  analysis  into 
the  evaluation  of  environmental  restoration  projects.  It  has  done  that  by  providing  some  overview  discussion 
of  environmental  planning  and  risk  and  uncertainty  analysis.  It  has  considered  generic  and  specific  potential 
sources  of  uncertainty  likely  to  be  encountered  in  the  environmental  planning  process.  A  simple  example 
has  shown  the  feasibility  of  developing  low  cost  methods  for  using  some  risk-based  analysis. 

At  least  two  subsequent  steps  appear  warranted.  The  first  and  most  important  is  to  expand  the 
discussion  of  introducing  risk  and  uncertainty  into  environmental  restoration  analysis.  The  attitudes 
mentioned  at  the  opening  of  this  section  can  only  be  addressed  as  the  education  and  debate  processes 
proceed.  This  report  may  find  some  role  in  furthering  that  education  and  debate. 

The  second  step  would  appear  to  be  to  demonstrate  the  feasibility  and  purpose  of  risk  and  uncertainty 
analysis  by  applying  it  to  an  on-going  study  effort.  Nothing  succeeds  like  success,  it  has  been  said.  And 
nothing  will  advance  the  cause  of  risk-based  decision-making  in  environmental  restoration  projects  as  well 
as  a  successful  demonstration  of  its  ability  to  contribute  positively  to  the  decision-making  process. 
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APPENDIX  A: 

MOTTLED  DUCK  RISK  MODEL 
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INTRODUCTION 

The  risk-based  analysis  presented  in  Chapter  Six  is  based  on  the  spreadsheet  model  described  in  this 
appendix  and  shown  at  Figure  A- 1.  The  model  was  built  using  Excel  v.  5.0  and  @RISK  v.  3.1.  It  was  built 
to  demonstrate  the  feasibility  of  doing  such  an  analysis.  As  such,  there  are  redundancies  and  inefficiencies 
that  could  be  eliminated  in  future  versions.  Or,  the  basic  rationale  could  be  adapted  to  existing  HEP  or  ECO- 
EASY  software.  The  explanations  that  follow  assumes  a  basic  familiarity  with  the  structure  of  an  HSI 
model. 


The  example  discussed  below  is  a  single  iteration  of  a  simulation.  The  simulations  used  for  this 
report  consisted  of  1,000  iterations.  Each  iteration  begins  with  a  new  set  of  randomly  generated  values  for 
the  habitat  variables.  These,  in  turn,  result  in  a  different  set  of  output  values. 

THE  MODEL 

Rows  3  through  24  reproduce  the  suitability  graphs  found  in  the  U.S.  Fish  and  Wildlife  Service 
(FWS)  HSI  model  for  the  mottled  duck.  Column  A  presents  the  percentage  values  that  correspond  to  the  Si’s 
for  each  of  the  five  habitat  variables  contained  in  columns  B  through  H.  A  more  efficient  model  would 
simply  reproduce  the  function.  For  example,  all  of  the  simple  linear  functions  could  be  replaced  by  a  simple 
function  of  the  form:  y  =  mx  +  b.  The  table  format  is  easier  to  follow,  so  it  was  used  for  demonstration 
purposes. 

Rows  27  through  33  present  values  for  classes  that  comprise  habitat  variables  V3,  V5,  and  Vg  under 
the  without  project  condition.  The  Sis  for  these  variables  are  weighted  sums  of  the  four  classes  of 
conditions.  Definitions  of  the  classes  and  details  of  the  weighting  function  can  be  found  in  the  mottled  duck 
HSI  model.  The  classes  are  defined  as  random  variables.  For  example,  cell  B32,  class  4  for  V3,  is  a  uniform 
distribution  ranging  from  0.9  to  1 .  Class  3  for  V3  is  1  minus  the  class  4  variable.  In  this  iteration  there  was 
94  percent  Class  4  and  6  percent  Class  3  for  the  V3.  There  were  no  observed  class  1  or  2  conditions  for  V3 
in  the  Corps  study  upon  which  this  analysis  was  built. 

Rows  34  through  40  repeat  the  process  for  with  project  conditions.  The  difference  this  time  is  that 
the  percent  of  each  class  observed  in  the  project  area  will  vary.  For  example,  consider  the  column  C  values 
for  V5.  Without  the  project  this  variable  comprises  classes  2  and  3  only.  With  the  project  it  comprises 
classes  3  and  4  only.  Changes  like  these  reflect  project  impacts. 

Rows  42  through  51  present  measurements  of  the  habitat  variables  displaying  the  without  project 
condition  values  in  column  B  and  the  with  project  values  in  column  D.  These  values  are  all  subject  to 
uncertainty  and  are  estimated  using  probability  distributions  that  reflect  this  uncertainty.  In  this  example 
uniform  distributions  were  used  for  simplicity  for  all  variables.  Variables  and  V,  must  logically  sum  to 
one  so  once  one  variable  is  randomly  determined  the  other  is  known. 


A-1 


B  I  C  I  D 


Figure  A-1:  Mottled  Duck  HSI  Model  with  Risk  Analysis 
Rows  3  through  24  display  linear  suitability  index  graphs 
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Rows  27  through  40  allow  for  input  of  without  and  with  project  condition  values  for  habitat 
values  based  on  class/step  functions  rather  than  linear  suitability  graphs. _ 


Without  Project 


Enter  %  of  each  class  as  decimal 


class  1= 
s  2  = 
class  3  = 
class  4  = 
Sum 


With  Project 


%  for  V3 

%  for  VS 

%forV8 

0 

0 

0 

0 

0.92988066 

0 

0.061555 

0.07011934 

0.040817 

0.938445 

0 

0.959183 

1 

1 

1 

Enter  %  of  each  class  as  decimal  ] 

%forV3 

%forV5 

%forV8 

0 

0 

0 

0 

0 

0 

0.0341965 

0.08156815 

0.0315256 

0.9658035 

0.91843185 

0.9684744 

1 

1 

1 

class  1- _ _ _ ^ ^ ^ _ 

class  2  =  ^  _2. _ 2- _ 

classic  ~  0.0341965  0,08156815  0.0315256  _ 

class  4  =  ^  ^  0.9658035  0.91843185  0.9684744 _ 

Sum _ _ ! _ ! _ ! - ; - ^ — 

Rows  42  through  51  present  measurements  of  the  habitat  variables  and  their  corresponding  Si’s. 
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[Rows  53  through  56  calculate  model  components,  (W/  and  W/O).  Life  requisites  are 
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Columns  C  and  E  present  the  suitability  indices  for  the  habitat  variable  values  presented.  The  SI 
values  for  variables  V3,  V5,  and  Vg  are  based  on  the  values  presented  in  rows  29  through  40  above.  Notice 
that  most  SI  values  differ  between  the  without  and  with  project  conditions. 

Model  components  and  life  requisites  are  calculated  in  rows  53  through  59.  The  components  in  rows 
53  through  56  are  functions  of  the  habitat  variables.  The  specific  details  of  these  functions  can  be  found  in 
the  HSI  model  and  are  discussed  briefly  in  Chapter  Six.  The  life  requisites  are  functions  of  the  components 
and  habitat  variables.  These  are  all  straightforward  calculations  or  equalities,  e.g.,  the  food  life  requisite  is 
simply  the  SI  of  V7.  Life  requisites  are  shown  in  rows  57  through  59. 

The  HSI  is  defined  as  the  minimum  of  the  three  life  requisite  Sis.  Under  the  without  project 
condition  in  the  example  this  would  be  the  cover  life  requisite  of  0.29  (cell  B57).  Under  the  with  project 
condition  it  is  the  food  life  requisite  of  0.70  (cell  D58).  Thus,  in  this  example  the  HSI  improves  from  0.29 
to  0.70  as  a  result  of  the  project,  a  change  of  0.41  as  shown  in  cell  F61 . 

Row  62  tests  the  sensitivity  of  the  HSI  to  the  decision  rule,  i.e.,  choosing  the  minimum  life  requisite, 
by  taking  the  average  of  the  three  HSl’s.  The  with  project  HSI  improves  from  0.66  to  0.84  a  change  of  0.18. 
A  third  decision  rule,  the  cube  root  of  the  product  of  the  three  life  requisites,  yields  values  of  0.58  and  0.83 
for  a  change  of  0.24,  as  shown  in  row  63. 

The  habitat  units  corresponding  to  each  of  these  three  decision  rules  are  presented  in  rows  64,  65  and 
66.  For  simplicity  it  was  assumed  the  mitigation  area  consisted  of  1 ,000  acres.  Thus,  the  HSI  of  cell  B61 
times  1,000  acres  yields  287  HU’s  (cell  B64)  without  the  project  and  700  HU’s  (cell  D64)  with  the  project 
for  a  change  in  HU’s  (cell  F64)  of  413  HU’s.  Row  64  presents  results  using  the  model  decision  rule  while 
rows  65  and  66  present  alternative  results. 

The  Corps  project  used  for  this  example  did  not  use  a  true  incremental  cost  analysis.  It  basically 
used  a  single  increment  of  1 ,000  acres.  Costs  were  assumed  to  be  determinate  so  this  demonstration  could 
focus  on  environmental  input  data.  They  could  have  easily  been  allowed  to  vary  to  reflect  the  uncertainty 
inherent  in  them.  Incremental  costs  for  this  single  increment  are  presented  in  cells  H64  through  H66.  Using 
the  model’s  decision  rule  of  the  minimum  life  requisite  as  the  HSI,  costs  are  $4,41 5  per  HU.  In  this  iteration 
the  other  rules  yield  incremental  costs  of  $10,220  and  $7,454. 
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